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ABSTRACT 

This paper examines various models for maintenance of a machine operating subject 
to stochastic deterioration. Three alternative models are presented for the deterioration 
process. For each model, in addition to the replacement decision, the option exists of 
performing preventive maintenance. The effect of this maintenance is to "slow* the 
deterioration process. With an appropriate reward structure imposed on the processes, 
the models are formulated as continuous time Markov decision processes, the optimality 
criterion being the maximization of expected discounted reward earned over an infinite 
time horizon. For each model conditions are presented under which the optimal mainte¬ 
nance policy exhibits the following monotonic structure. First, there exists a control limit 
rule for replacement. That is, there exists a number /* such that if the state of machine 
deterioration exceeds i* the optimal policy replaces the machine by a new machine. 

Secondly, prior to replacement the optimal level of preventive maintenance is a nonin¬ 
creasing function of the state of machine deterioration. The conditions which guarantee 
this result have a cost/benefit interpretation. 

INTRODUCTION 

There is a substantially large collection of papers in the machine repair/main¬ 
tenance/replacement literature. Two fairly comprehensive surveys are McCall [22] and Pier- 
skalla and Voelker [23]. Specific articles of interest are Barlow and Proschan [ 4 ], Derman [8], 
[91, [10], Klein [17], Kolesar [18], Kalymon [15], Thompson [29], Kamien and Schwartz [16], 
Ross [25], and Rosenfield [24]. An interesting semi-Markovian treatment of shock models is 
given by Feldman [11]. The focus of these models has been on the replacement decision. In 
such models, gradual operating deterioration (or 'wearing out") is not subject to control. The 
model in [16] differs somewhat in that preventive maintenance can be applied to decrease the 
failure probability over time. 

In this paper we formulate three machine maintenance models that in addition to the 
replacement decision, incorporate the option of performing preventive maintenance. The effect 
of the maintenance is to "slow" the rate of machine deterioration. Each model presents a 
different version of the nature by which maintenance action afTects the deterioration process. 

Results 

For each model conditions are given under which an optimal maintenance policy exhibits 
the following monotonic structure: 
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1. Control limit rule for replacement. There exists 0 < /'* < +o° such that when the state 
of the machine i > /*, the machine is to be replaced. 

2. Decreasing preventive maintenance. For 0 < / < /* denote an optimal maintenance 
level by a*(i). Then a* is a nonincreasing function of /. (Compare to [16).) 

The technique employed to obtain the above results first involves transforming the con¬ 
tinuous time Markov decision process (CTMDP) to an equivalent discrete time Markov process 
(DTMDP). The infinite horizon problem will then be approached via the associated finite hor¬ 
izon discrete time process, thus permitting induction arguments on the "ff-period" return func¬ 
tions. We then use the fact that the n-period return converges to the infinite horizon return 
function (see Bertsekas [5], Denardo [7J, Ross [26)). This technique has been successfully 
employed by others, notably Lippman [191, (201 in queueing optimization. 

Papers which focus on other aspects of optimal control of the machine maintenance/repair 
problem are Albright [ll, Crabill [6), Goheen (12), and Winston [321, [33). 

MODEL DESCRIPTION-MODEL 1 

We consider a machine operating over an infinite time horizon. The state of the machine 
is represented by / € 5 — S 0 U {/}, where S 0 — {0,1,2, ...}. State zero represents a new 
machine, state / a failed machine, and increasing integer state values represent increasing stages 
of machine deterioration. 

The action space is denoted by A and consists of triples (a,, a 2 , a 3 ) € R } and is defined 
as follows: a 3 « 0 or 1 according to whether the machine is not or is replaced by a new 
machine. If a machine in state i is replaced by a new machine, the transition i — 0 takes place 
instantaneously. When a 3 - 0, the transition / — / + 1 occurs at (exponential) rate X|(/, a t ) 
and the transition to failure i — / occurs at rate X/U a 2 ). The term "rate" is used here in the 
usual sense, see Ross [261. For example, let p u+ , (o|) be the transition probability for the 
transition i — i + 1 given action <j) and suppose the transition time probability out of state / is 
exponential with mean 1/x(i, a^. Then X|(f, a i) = kii, a x )p ilJr \ ia\). Finally, if the machine 
is in the failed state, it must be immediately replaced (o 3 - 1 when i - /). a, and a 2 are 
viewed as maintenance actions in a sense to be made precise below. 

The reward structure is specified as follows. A lump sum cost C is incurred when the 
machine is replaced by a new machine. If the machine fails while in service a lump sum cost 
F > C is incurred for replacement. If the state is i € S 0 , the machine earns revenue at rate 
r(i, a) per unit time, a - (a t , a 2 , 0). The optimality criterion is the maximization of 
expected discounted revenue earned over an infinite time horizon. For the continuous 
discounting of cash flows denote the discount factor by a € (0, 1). 

Notation and Definitions. For a real-valued function g defined on the nonnegative integers, 
let Ag(i) - gii) - g(i - l), i ^ 1, and fi 2 g(i) - g(i) - 2 gii - 1) + g{i - 2), / ^ 2. Thus, 
g is nonincreasing (nondecreasing) iff AgU) < 0 O 0) and convex (concave) iff A 2 g(i) > 0 
« 0). 

Let / be a real-valued function defined on C x D £ R". f is supermodular (submodular) 
on D if for d\, d 2 € D, fic, d\ - d 2 ) + f(d, d\ - d 2 ) > «)/(c, d\) + fid, d 2 ), where * 
denotes component-wise minimum and - denotes component-wise maximum. / has antitone 
( isotone) differences on C x Dif for C| ^ c 2 and d\ < d 2 . 
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/(c,. d 2 ) -/(c,. </,) > «)/(Cj. </ 2 ) -f(c 2 , d t ). 


Let A' ~ A - [a € A Uj — 1}. 

Define A - sup {2\|(£ a\) + k,ii, a 2 )}. 

i.a 


We make the following assumptions for Model 1: 

(Al) A < +<» 

(A2) A is a compact subiattice of R } (under the usual partial ordering and relative topol¬ 
ogy). 

(A3) r is nonincreasing convex in /, nonincreasing in (a\, a 2 , 0) € A and has antitone 
differences on S 0 x A Also, r > 0. 

(A4) x, and k r are each nondecreasing concave in i and nonincreasing on A'. X| has iso¬ 
tone differences in (/, a\) and k f isotone differences in (/, a 2 ). 

(A5) X,, kf, and rare each continuous on A'. 

REMARKS: Given state / and action a 2 , the probability that the machine will fail in a 
small time dt is k,ii, a 2 )dt. In this sense we think of X^ as a failure rate function. Similarly, 
X| is a measure of the tendency of the machine to "wear out." The fact that the rates X] and X f 
are nondecreasing in i will play a major role in establishing a control limit rule for replacement 
(compare to Derman’s IFR conditions (8i). The monotonicity of X| and kj in (a u a 2 , 0) 
makes precise the notion of viewing the action as "preventive maintenance." 

The concavity assumptions on X| and X f can be given the interpretation that when the 
machine is new there is a strong tendency for wear-out and failure to increase sharply. As the 
machine breaks in, this rate of increase declines. This concavity assumption is used to prove 
the optimality of decreasing preventive maintenance. 

From (A3), letting b > a we have r(j + 1, a) - r(i + 1 ,b) > r(i, a) — r(i, b) which 
says that increasing the level of maintenance from a to b incurs a greater cost increase in state 
/ + 1 than in state /. Not only does the cost increase but the benefit decreases since from (A4) 
X|(/ + 1, a) — X|(i + 1, 6) < X|(i, a) - Xi(i, b) (and similarly for X/). 

THEOREM 1: There exists an optimal stationary policy for Model 1. 

PROOF: Note that from (A2), (A3), and (A5) the reward function is bounded. It is 
then trivial to verify that the conditions specified in [7] are satisfied. (The functional operator 
H g defined in that paper turns out to be a two stage contraction due to the possibility of instan¬ 
taneous transitions i — 0. For unbounded return functions see Lippman 121].) 

Functional Equations 

Using the method described in Lippman [19] (see also Kakumanu [14], Serfozo [28], 
Winston [32], Howard [13], Anderson [2]) we can transform the continuous time process to an 
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equivalent discrete time Markov decision process (DTMDP). Denote the optimal return func¬ 
tion for the n-period horizon in the DTMDP by V„, n < +°°- 

Then 

(1) W = 0 

and for n > 1, 

| Max \T„(C, 0), T„U)) i*f 

(2) VJ') “ | jrJF, 0) i - /' 


For x € {C, F\, 

T 0 (x, 0)5 0 

and for n > 1, 

(3) T (x , 0) — Max {/■ (0. a) — Jr (a + X|(0, fl|) + X/(0, a 2 )) 

+ X,(0, +X r (0, a 2 ) T„- X {F, 0) 

+ (A - X,(0, fl t ) - X,(0. a 2 )) K„_,(0))/(A + a) 

T q U) = 0 

T.(i) - Max 7(4 a, F„_ t )/(A + a) 

( 4 ) 7(4 a, K,_|) » r(4 <z) +Xj(4 0 t)F„_|(/ + 1) + 

X/(i, a 2 ) r„_,(F. 0) + 

(A - Xi(i. fl t ) - X/(i. a 2 )> 


(For „.+<»we write K - K L- D. Equations (2) assume that a failed machine must 
be replaced The present of the lump sum rewards -C and -F requires special consideration 
in the discrete time recursions resulting in their somewhat complicated ap^rance invohjing 
the functions T n . For a discussion on the treatment of lump sum rewards see Serfrao 1281 
Anderson [2]. In (2), T„(C. 0) is the return if the replacement action is chosen, T„(F, 0) is 
the return if an in service failure occurs, and T„(i) is the return otherwise. 

We denote the optimal action in state i by o*(4 n) and resolve tiesi in (3) by defining 
a*(i.n) - inf {o € A'\J(i, a, F„_,) - Max 7(4 a, which is well-defined since 7is con¬ 

tinuous on A' and A is compact. 


Finally, if a tie occurs in (2) we define V„(i) - T„(i). 

LEMMA 1: ForO < n < +» and i € S 0 , T„(F. 0) < V„ (i). 

PROOF: Equation (2) implies F„(») > T n (C, 0) > T*(F, 0) since F > C. p 


LEMMA 2. For 0 < n < +°°. K « nonincreasin» in i € S 0 . 
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PROOF: We prove the lemma for finite n (by induction). The infinite horizon case fol¬ 
lows by taking limits (V„ —■ F). For n - 0 the result is trivial. For n > 0 it suffices to show 
that AT„(i) < 0. Define a - a*(i) « (a,,a 2 , 0). From (3), (4), and the definition of a, 

(5) (A + a) lT„(i) - T„U - l)] < JU, a, F„_,) - JU - l, a, V n _ x ) 

< AF fl _,(» + 1)X,(/, a,) 

+ AF„_,(/) [A - X)(i - a, a |) - X/(i - 1, a 2 )] 

(6) < 0. 

Inequality (5) follows from \/(i. a 2 ) > X jU - 1, a 2 ) and T n ~ x (F, 0) - y„- X U) < 0 (Lemma 
1). Inequality (6) follows from the definition of A and the inductive hypothesis. □ 

THEOREM 2 (Control limit rule for replacement): There exists /*, 0 < i* < +°°, such 
that for i > r* a*U) — 1 (i.e., the optimal action is replacement). 

PROOF: From (2) it suffices to show that T is nonincreasing on S 0 . In the proof of 
Lemma 2 we showed that A T„U) ^ 0 for finite n. Using A F(/) ^ 0 (Lemma 2), the proof 
that A T„U) ^ 0 holds in the same way for n - +“>. Now set i* — inf (i: T(C, 0) > r(/)). D 


In order to establish the optimality of nonincreasing preventive maintenance we first need the 
following lemma. 


LEMMA 3: For 0 < n < +«>, F„ is convex in i. 

PROOF: The convexity of V will follow from that of V„(n < +<») by taking limits on n. 
For n < +°° we employ induction. Since the maximum of convex functions is convex, from 
(2) it suffices to show that T n is convex. Since F 0 = 0 the convexity of F 0 is trivial. Assume 
then that F„_, is convex. To show that T„ (and, therefore, V„) is convex we need A 1 T„{i) - 
(r„(i) - r„(i - D) - (7* b (/ - 1) - T„(i - 2)) > 0, i > 2. Define a-a*(/-l, 
n) - (a Xt a 2 ,0). From (3) we have 

(7) (A + a) 17 ;(/) - T„U - 1)] > J(i, a, V „_,) - JU - 1, a, F„_,) 

(8) (A + a) l T„U - 1) - T„U - 2)1 < JU - 1, a, F„_,) - /(/ - 2, a, F„_,). 

From (7) and (8) in order to establish ^T„U) > 0 it suffices to prove that A 2 JU, a) = 
JU a, F n _,) - 2 JU - 1. a, F„_,) + JU - 2, a, F„_,) > 0. 


From (4), 

(9) 

( 10 ) 

( 11 ) 

( 12 ) 

(13) 

(14) 


A 2 JU, a) — rUa) - 2 rU - 1, a) + r(i - 2, a) 

+ X(((, a x ) y„~ x U + I) 

+ AK„.,(i)(A - 2X,(i — 1, a,)l 
+ A F„-|(i - DI- A+XjO-2, ai)l 
4- T„_ X (F, 0) Ikfii, a 2 ) ~ 2\f(i — 1, a 2 ) + X/(t ~ 2, a 2 )l 
+ F„_|(/) [—X/(/, a 2 )l 
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U5) + K„-|(i - 1) l2X r (i — 1, a 2 )l 

(16) + V„- { ti ~ 2) l-kfil - 2, a 2 )] 

Consider the last four terms of the above sum ((13)-(16)). From Lemma 1, T„.\(F, 0) ^ 
V„_\(i - 1) and using the concavity of A, we see that the sum of (13M16) is greater than or 
equal to 

(17) AK,_,(/)[—A,(/, a 2 )l + 1) M'~ 2 * 

Replacing (13)-(16) by (17), using the convexity of r, the definition of A and the induction 
hypothesis that V n _ i is convex (AK„_i(i) > AK„_i(/ — 1)), we see that 

A 2 J(i, a) > AV„^ (/- 1) [A,U flj) — 2X|(f — 1, a,) + \,(» - 2, a,) 

+ k,(i - 2, a 2 ) - A f U, o 2 )l 

> 0 

where the last inequality follows from the concavity of A| and the monotonicity of Ay. □ 


THEOREM 3: Prior to replacement, the optimal preventive maintenance level is a nonin¬ 
creasing function of /, i.e., for 0 < / < /*, a*(i) > a*(i + 1). 

PROOF. From Theorem 6.2 of [30] and assumption (A2) it suffices to show that (1) for 
each /, J is supermodular in a € A' and (2) J has antitone differences on A' x S 0 . (1) follows 
from the supermodularity of r and the fact that J — r is a separable function of a\ and o 2 . 

Let a - (a!, a 2 , 0) < b - (&i, b 2 , 0) € A'. Define A, a() = J(i + 1, b, V) + J(i, a, K) - 
JU + I, a, V) - J(i, b. V). 

Then, 

(18) A (afc - r(i + 1,6) + r(t, a) - r(i + 1, a) - rU, b) 

(19) + AF(i + 2) [A|(i + 1, b\) — A,(t + 1, a ,)1 

(20) + AP(/ + l)[A|(t, a) - A|(», 6)1 

(21) + (T(F. 0) - VU + I)) [k,ii + 1, b 2 ) - k f (i + 1. a 2 )l 

(22) + (T{F, 0) - V(i)) [k f U, a 2 ) - A r U, b 2 ) 1 

(23a) < AVO + 1) [Ai(/ + 1. b,) - A,(i + 1. a ,) + A,(». a,) - A,U 6)1 

(23b) + (T(F, 0) - V(i + D) Ikfii + 1. b 2 ) - k f (i + 1. a 2 ) 

+ kf(i, a 2 ) ~ A f(i, 6 2 )1 

(24) < 0. 

The inequality yielding (23a) and (23b) results from the following: From (A3), summand (18) 
is nonpositive; use the convexity of V and the fact that A i is nonincreasing on A' to obtain 
(23a) from (19) and (20). Now use Lemma 2 and the fact that A, is nonincreasing on A' to 
obtain (23b) from (21) and (22). 


Inequality (24) follows using Lemma 1, and assumption (A4) that Ai and A 2 have isotone 
differences. 


□ 
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This completes the specification of the structure of the optimal maintenance/replacement 
policy for Model 1. Note that the results hold for the special cases X s = 0, or X, = 0. Also 
observe that from (4), J(i, a, V) — r(i, a) 4- X ( (/, a,) A KG' + 1) + X f U, a 2 ) ( T(F 0)- 
V(i)) + A V(i). Thus, if r is convex in a, and X, and X f are concave in a, J will be convex in 
a. (Recall Lemmas 1 and 2.) In this case, a* is a bang-bang policy for / € S 0 : 

a* — sup {a 6 A'\ or a* - inf {a 6 A'). ( A ' is compact by assumption (A2).) X] concave 

nonincreasing in a\ means that small increments in a\ above inf A' have little effect in decreas¬ 
ing X, (and similarly for a 2 and X f ). However, r convex nonincreasing in a implies that the 
increase in cost for such an increment is great. Whether a*(0) — sup/4' or a*(0) — inf A' will 
depend on how effective preventive maintenance is against protecting the new machine relative 
to the cost of the machine. 

MACHINE MAINTENANCE - MODEL 2 

The specification of Model 2 is identical to Model 1 regarding the state space, reward 

structure, and optimality criterion. However, Model 2 differs in the nature of machine 

deterioration. Preventive maintenance is represented by a pair a — (a t . a 2 ,0) € A £ R y . 
The transition / — / (failure) occurs at rate X,(/. a 2 ). Also, the machine may deteriorate to 
state i + j from state i at rate X O'. a\), j € {1.2. ...). 

Suppose the previous assumptions in Model 1 on A, r and X t hold here. In addition, we 
assume that X(y, •) is nonincreasing and continuous on A' for j € S 0 . Also, assume that A = 
sup {Ix(/, a|) + Xjii, ay)) < +°°. Then routine application of the techniques used for Model 
1 establishes that the optimal maintenance/replacement policy for Model 2 exhibits the same 
monotonic structure. Since the proofs are completely straightforward we omit the details. 

MACHINE MAINTENANCE-MODEL 3 

Again Model 3 differs from Model 1 only in the nature of the deterioration process. 
Specifically, denote an action by a = (a t , a 2 , a 3 ) € A C R 3 . a 3 = 1 indicates replacement as 
before. Suppose a } = 0. Then failure / —» / occurs at rate K f (i, a 2 ) as before. In addition the 
machine may also deteriorate from state /'to a state / + m(i, fl ]( p) at rate \dF(p). Here pis a 
random variable taking values m a set P and having distribution t Let Z denote the set of 
integers. Then m is a map nr. S 0 x A x P — Z. We think of p € P as a measure of the mag¬ 
nitude of a randomly occuring shock. The time between shocks is exponential with mean 1/X. 
The three variables i, oj, and p together determine the subsequent "damage" m(/'. a^p) to the 
machine. 

We make the following assumptions: 


(Bl) A = sup {x Ai, o 2 )} + X < +°° 

i.a p 

(B2), (B3) same as (A2) and (A3), respectively. 

(B4) The conditions on X f are the same as stated in (A4). 

(B5) The function m is nonincreasing in a\ ('preventive maintenance"), is nondecreasing 
in i € S 0 , and has isotone differences on S 0 x A. 
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(B 6 ) m, X r , and r are each continuous on A. 

These assumptions are to be interpreted in a manner similar to the discusssion following 
(A1)-(A5) of Model 1. 

It is easy to establish Theorem 1 for Model 3. We now present the recursions for the 
present model. 

Recursions for Model 3 

The DTMDP recursions equivalent to the CTMDP are the following: 

(25) y 0 U) = 0 
and for 1 < n < +°°, 

( Max{T„(C,0), T„(/» / * / 

(26) V„(i) - ( j n ( F o) /- / 

For x € (C. F}, 

T 0 (x 0 ) = 0 

and for n > 1, with A' - A - [a € A U 3 -* 0), 

T n {x, 0) - Max {r(0, a) - x(a + X + X/(0, a 2 )) 

a€/4' J 

+ X J p 0, fl|, p)dF(p) + X / (0. a 2 )T„~\(F, 0) 

+ (A - X - X/(0, a 2 )) K b _|( 0))/(A + a) 

T 0 (i) = 0 

for n > 1 , 

(27) T„(i) - Max JU, a, F„_,)/(A + a) 

a€A 

(28) JU a. F n _|) - r{i, a) + X J p (/ + mU, a u p)) dF(p) 

+ k f U a 2 ) T n .\(F, 0) 

+ (A - X - k f (i, a 2 )) 

As before for n — +<» we write — V and T x — T, and a*(i, n) denotes the optimal 
maintenance action in state i. 

LEMMA 4: For 0 < n < +<» and i € S 0 , 

T„(F, 0) < V„(i). 

PROOF: The proof is similar to the proof of Lemma 1. □ 

LEMMA 5: For 0 < n < + 00 , y n is nonincreasing in i € S 0 . 
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PROOF: As before, the infinite horizon case will follow from the finite horizon result. 
The result is trivial for n - 0 since = 0. Assume A V„~\(i) < 0 for i € S 0 . From (26) it 
suffices to show A T„(i) < 0. Let a * a*0‘, n) - (a!, a 2 , 0) € A'. 

From (27), 

(29) A T„(i) < JU, a, K„_,) - JU - 1, a, F„_,) 

*• A/■(/', a) 

+ X J* p F n _, (/ + m(/, a,, p)) - F„_|(/ - 1 + m(/' - 1, a,, p)) dF{p) 

+ k f ti a 2 )[r„_,(F. 0 ) - K„_,(i — 1)] 

- x,(; - l, fl j) {r ff _,(F, 0) - v„^a - 1)] 

+ (A - X)A V„~\(i) 

< A/■(/, a) + 

X/. J* + m(i, a,, p)) - y„-\(i - 1 + m(i - 1, a\, p))]dF{p) 

+ (A - X)A y„-\(i) 

(30) < 0. 

Inequality (29) follows from Lemma 4 and the monotonicity of \ f in /. To obtain the last ine¬ 
quality, (30), use (A3) on r, (B5) on M, and the induction hypothesis that is nonincreas¬ 
ing, along with the definition of A. D 

THEOREM 4 (Control limit rule for replacement): There exists /*, 0 < /* < +<» such 
that for /' > i* ,a*(i) = 1 (replacement). 


PROOF: The proof is similar to the proof of Theorem 2. □ 

LEMMA 6 : For 0 < n < -H», V„ is convex in i. 

PROOF: As in Lemma 3, it suffices to show that T„ is convex. K 0 = 0 so assume that 
F (l _, is convex. Let a — a*U - 1, n) — (a ( , a 2 , 0) and 

A 2 JU, a) = JU, a, K„_,) - 2 JU - l, a, K„_,) + J(i - 2 , a, F„_ t ). 


From (27), 

A 2 7;</) ^ A 2 JU, a) 

(31) 

- A 2 r(/, a) 


+ X [y„-\(i + m(i, a,, p)) - 2V„. 

(32) 

+ y„-\(i - 2 + mU - 2, a 1( p))l dF(p) 

(33) 

+ A J X / (( a 2 ) r n _,(F, 0) 

(34) 

+ K„_,(i)lA - X - X/(/, a 2 )] 

(35) 

+ K B _,(/ - 1) {-2A -f 2X + 2X/(/ - 1, ff 2 )l 
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(36) + F„.,(/ - 2) [A - A - kf(i - 2, a 2 )]. 

From (B3), A 2 r(i, a) > 0. Define m (l = / + m(/. d|, p), m\ — i — 1 + m(i - 1, a;, />), and 

m 3 - / - 2 + mU - 2, a,, /?). From (B5) m () ^ m { ^ m 2 and m concave in / »*• m 0 - 

Wi $ Wi — m 2 . Since K„_, is convex (the inductive hypothesis) and nonincreasing (Lemma 
5), F„_,(m 0 ) - y„-\(m |) > F„_|(mi) - y„^,(m 2 ) which establishes that the integrand in (32) 
is nonnegative. 

Finally, the proof that the sum of the last four terms (33M36) is nonnegative is the same 
as in Lemma 3. Thus, A 2 J(i, a) > 0. n 

THEOREM 5: Prior to replacement the optimal preventive maintenance level is a nonin¬ 
creasing function of /. 

PROOF: As in the proof of Theorem 3 it suffices to show that (1) for each /, J is super- 
modular in a € A' and (2) J has antitone differences on A' x So. (1) follows as before. To 

establish antitone differences we first invoke (B3) on r. Also, the proof that J has antitone 

differences in i and a 2 is the same as in Theorem 3. It thus suffices to prove that the integrand 

in (28) exhibits antitone differences in i and a |. Let a, < 6 |. Define 

ni\ = / + 1 + m(i 4- 1, b\, p ), m 2 = / 4- 1 + m(i + 1, a |, p), my = i + m(i, b\, p), and 

m 4 — / + m(i, a,, p). From (B5) m i < m 2 , my ^ m,, my ^ m 4 , m 4 ^ m 2 . Define 

g(i, U|) = y(i + m(i, a\, p)). Then 

[g(t + 1, b\) - g(i + 1, u,)l - IgO, b\) - g(i, o,)] 

- [F( m ,) - y(m 2 )I - (F(m ? ) - U(m 4 )I ^ 0. 

The inequality follows from the fact that V is nonincreasing convex and the fact that 
m 2 - m\ < m 4 - m } (namely (B5) which assumes that m has isotone differences in (/, a,)). 
This completes the proof. n 
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A NEW BIVARIATE NEGATIVE BINOMIAL DISTRIBUTION 
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ABSTRACT 

A new bivariate negative binomial distribution is derived by convoluting an existing 
bivariate geometric distribution: the probability function has six parameters and admits of 
positive or negative correlations and linear or nonlinear regressions. Given are the mo¬ 
ments to order two and. for special cases, the regression function and a recursive formula 
for the probabilities. Purely numerical procedures are utilized in obtaining maximum 
likelihood estimates of the parameters A data set with a nonlinear empirical regression 
function and another with negative sample correlation coefficient are discussed. 


1. INTRODUCTION 


In this paper we develop a new bivariate negative binomial (bnb) distribution by convolv¬ 
ing a certain bivariate geometric distribution. The univariate negative binomial with parameters 
v > 0 and 0 > 0 is defined (Johnson and Kotz [6)) as the distribution of a random variable 
(r.v.) X for which 


( 1 ) 

Pr[X - x) 


T(x +v) 1 


x!I>) | 1+61 
x + v — l| 


1 +0 


\x 4- V — 1 


1 +0 


1 +0 


x- 0. 1. 2. ... 


shall be taken to be defined as the ratio of gamma functions; the charac- 


where 
teristic function is 

£V'*]- [l +p(i - *>")]-*. 


The mean and variance of X are v0 and p0(l +0), respectively. For v ■* 1 we have the 
geometric distribution. We shall find it more convenient to use probability generating functions 
than characteristic functions because of the discrete nature of the random variables we consider. 
The correspondence between characteristic functions and probability generating functions is 
effected by letting u - e u which gives 


'This research was performed when the author was affiliated with the United States Air Force Academy. Colorado 
Springs, Colorado 
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( 2 ) 4>(u) = £[«*] = [ 1 + 0(1 - m ) 1 ". 

Another common representation is to let 0 - p/( 1 - p), or equivalently, p - 0/(1 +0), 
in (1) and so 

(3) Pr[X - xl » (1 ~ pY P*. x- 0,1,2.... 

x ir (v) 


(4) 

<t> in) = 

1 — pu 


1 + t- 2 — (1 - u) 



1 ~ P 


1 - p 


This latter representation is referred to as a negative binomial distribution with parameters v 
and p. We use both representations throughout. 

The probability function of a bnb distribution (Mardia [91) (or negative multinomial 
(Johnson and Kotz, (61) is 

(5) Pr[X = x, Y = vl = - r (1 - p - qY p x q\ x, y - 0, 1, 2, ... 

x !.y !r (v) 

where v > 0, p *= 0/[l + (a + 1)0], ? = op, 0 < p < 1, 0< q < 1, and 0 < p + q < 1. 
The probability generating function is 

(6) <b (w, v) = £[m*vM — [1 + 0 (l - u) + a0 (1 - v)]'". 

Guldberg [5] introduced this distribution and Bates and Neyman [3] fitted it to several data 
sets. We designate this bnb distribution as G - B - N{a, 9, v). The distribution admits only 
of positive correlation and linear regressions. 

Certain data sets do not exhibit empirical regressions which are linear nor do some data 
sets show positive correlation and so it is natural, for these cases, to work with a bivariate pro¬ 
bability function which allows for nonlinear regressions or negative correlations or both. The 
classical Bates and Neyman paper exhibited empirical data which would seemingly be best fit by 
regression curves which were obviously nonlinear, and, consequently, their results were not 
entirely satisfactory. Furthermore, in Table 3 we provide some new bivariate data related to 
aircraft flight aborts which has a negative sample correlation coefficient. We would thus be 
reluctant to use a bivariate distribution which did not admit of the possibility of both negative 
correlation and nonlinear regressions to describe the observed abort phenomenon. These few 
instances (and several others we do not provide) indicate there is a need for a study of the pro¬ 
perties and potentialities of new multivariate negative binomial distributions which provide for 
more flexibility than those which have heretofore been discussed in the literature. We also iso¬ 
late some situations which call for even more general distributions than we consider herein. 
Before we perform the data analysis we discuss an inventory model which generates the new 
distributions and examine the resulting distributions in some detail. 

2. A STORAGE MODEL 

Consider the storage system consisting of two facilities, one of which is used to store 
commodity A alone, the other to store commodity B alone. At each epoch of time, 

n ” l. 2.facility 1 receives a random amount of commodity A , say a n , and facility B 

receives a random amount of commodity B , say b„. Take the a„ and b„ to be mutually 
independent, identically distributed random variables. Let the total of A and B at epoch n be 
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A„ and B„. The total A„ and B„ are related to the totals A n - X and B n -\ in the following 
fashion. Between epoch n - 1 and n the totals A n _ | and B„_ | may randomly be reduced to 
zero or remain intact. The manner in which A„-, and are carried over to the «th epoch 
determines storage characteristics. The system equations are 

A„ ” O n + £„ 

Bn “ b„ +£„ B n -\, 

and without loss of generality we take Ao = Bq *= 0. There are four cases of interest: 

(I) Pr (£„=“£„ =* 0) = a 

Pr (£„«<„« 1 )= <*« 1 - a. 

(II) Pr (*„=■ 0. = 0)= a 

Pr 1. 0) = b 

Pr (£„ = 1. £„ = 1) - d= 1 - a - b. 

(III) Pr (£„= 0. £„= 0) - a 

Pr (£„» 0. D- c 

Pr (f„- 1. t„ - D- rf - 1 - « - c. 

(IV) Pr 0. £„« 0)- a 

Pr (*„« 1. 0)- b 

Pr 0. £„ « 1) - r 

Pr (f„- 1. Z n - D- 1 - a - b - c. 

For example, (I) implies that the contents of facilities A and B are either both discharged or 
both kept in the time between epoch n - 1 and n. (II) implies that the contents of both are 
simultaneously discharged with probability a; the contents of B is discharged while those of A 
are kept with probability b\ and the contents of both A and B are kept with probability d. 
Finally, take both a„ and b„ to be geometric. Provided b + d < 1 , c + d < l,the system 
defined by the above system equations soon reaches steady state and is governed by a bivariate 
geometric probability distribution (Paulson and Uppuluri, {II)). If there are m identical sys¬ 
tems of this type and the quantities A and B are summed across the m systems, a bivariate 
negative binomial distribution results. Curiously, cases (I), (II), and (III) give rise to infinitely 
divisible bivariate distributions (as we show iater) whereas (IV) does not in general The 
analytical details, starting from the steady state system whose behavior is defined by a func¬ 
tional equation in the probability generating function, are given in the next section. 

3. THE NEW BIVARIATE NEGATIVE BINOMIAL DISTRIBUTION AND PROPER¬ 
TIES 

Paulson and Uppuluri [Ill showed that the bivariate r.v. (X, K) (identify X and Y with 
the steady state A„ and B „, respectively), where each element in the pair is defined on the non¬ 
negative integers, has a bivariate geometric distribution if its probability generating function, 
4>(u, v), satisfies the functional equation 
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(7) <6 (u. v) — ill | (u, l)i|( 2 ( 1 . v) (a + M> («, 1 ) + c<t> ( 1 , v) + dd> (u, v)) 


where 


<6i(w. 1) 
</< 2 (l. v) 


1 + - 

7 -P- - (1 ~ U ) 


1 - P 

1 + - 

r - 4 —(1 - v ) 

1 ~ q 


and a, b, c, d are all nonnegative quantities with a+b+c+d- 
and 0 < p < 1 , 0 < q < 1 . It is easy to show from (7) that 

( 8 a) <b(u, 1 ) - (1 + 0,(1 - u))' 1 . 

( 8 b) < 6 ( 1 . v) - (1 + # 2 0 - v))~>. 


1 , b + d < 1 , c + d < 1 , 


with 

(8c) 9 1 - p/K 1 - p) (a + c)J. 0 2 — q/\( 1 - q) (a + 6)1. 

We thus see that the marginals are geometric. Paulson and Uppuluri obtained the moments of 
( X. Y) in (7) to order two and showed that the correlation coefficient p varies over —.25 
$ p < 1 ; they also presented recursive formulae for determining the probability function. 


Consider the case b = c - 0 in (7). We have (suppressing the arguments of <b |(u. 1) to 
give ili i • ili \ (u. 1 ) and similarly for <6 2 ) 

(9) <6 (u. v)-ilifli 2 ((1 - d) + d<f>(u, v)) 

(1 - d)<li flii 
1 - dili |<(/ 2 

(10) - (1 - d)ili fli 2 (1 + dili i<6 2 + d jty 2 + ...). 

The inverse transform of < 6 (u, v), the probability function £|(x. y), say, may be obtained 
termwise from ( 10 ) since the resultant series converges uniformly and absolutely for all 
x, y — 0 . 1 , 2. ... (Titchmarsh, 112)). We obtain 

(11) £|(*. y) - (1 - d) (1 - p)p*(l - q)q y £ j x ^ “j (rfd - p) (1 - ff)^. 

where x. y - 0, 1. 2.Expansion of the combinatoric terms in (11) gives 

(12) g|(x, >>) — (I — d) (l — p)p x (1 - q)q y Fix + 1, y + 1; 1; d(l - p) (1 — q)). 


where Fla, 6 ; c; z) is the Gaussian hypergeometric series given by 


(13) 


F(a, 6 ; c;z) 


oo 

1+1 

/-I 


(fl) / -(6) > 


zi 

j'.’ 


and the term In), is defined by 


In), - “ nln + 1 ).,. (n + j — 1 ). 
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It can be shown that Theorem 12.2.3 of Lukacs [8) can be appropriately extended to yield 
the result that^lu. v) is infinitely divisible, or equivalently, #|(x, >0 is infinitely divisible (see 
also Block, Paulson, and Kohberger [4]. 

Thus, the v-fold convolution of g\(x, y) with itself yields a bnb distribution which we 
designate &,( x. y) where v need not be restricted to integer values. The probability generating 
function (u, v) of g„ix. y) is 

(14) <b„iu, v) - ((1 - d'bb &iY(\ - ebb 2 

- ((1 - dbb itl»2> , '(l + vdty fit 2 + - — — (dii> tfi 2 ) 2 + ...) 

which gives in the same manner as above 

(15) g„(x, y) — (1 - dY h\(x)h 2 (y) Fix + v, y + v ;v;z), 
where 

|*» + x — 1 

M*) ”1 x (1 ~ pY P* 

( v + y — 1 

y (1 - qY q y 

z - d{\ - p) (1 - q) 

and x. y - 0. 1. 2. It is clear by construction that the marginals of g„ (x, y), say ?*„(x) 

and j?>„(y), are negative binomial with 0 t ** p/iai 1 - p)) and 02 “ q/iai 1 - q)) as may be 
determined from (8) with b -* c - 0. 

The infinite divisibility of g v (x. y) is a result of considerable appeal in applications since v 
need not be restricted to integer values. 

We remark in passing that the distribution (15) is the discrete analogue to the Wicksell- 
Kibble bivariate gamma distribution. The k-variate extension may be readily derived as above 
and has probability distribution (in obvious notation) 

k 

£„(xi. x 2 . x k ) - Cik) + v . x k + . II <*-*»• 

i-i 

with 

* (*' + X J ~ *1 

c^)-n| Xi I (i - P'Yipj)*. 

Next we consider the case with b and c different from zero in (7). For notational con¬ 
venience, set <t>\iu, 1)-*>| and <£ 2 (1. v) - <b 2 . Denote the probability function corresponding 
to (7) by f\(x, y). Observe that when b - c - 0, /i(x, y) - gi(x, y). The (integral) m-fold 
convolution of f\(x, y) with itself has probability generating function <t>„(u, v), probability 
function / m (x, y), and marginal probability functions /* m (x) and fy m iy), respectively. With 
some minor calculation we find 
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(16) 


<A m («. v) 


(1 - dtothi 
1 - <M |t// 2 



c 

1 - d 


* 


m 


From (8) or (16) it is easily shown that 
(17a) <6 m (u. 1) - (1 + #,(1 - M))“ m . 

(17b) <M1. v) - (1 + 0 2 (1 - v))*. 

and accordingly, 

1 +«, 


(18a) 


fxm(x) 


T (j c + m) 


r(x + i)r(m) I i + », 


l 


(18b) 


Am<y) = 


r(y + m) 

r(y + nr(m) 


1 +»: 


1 + 0 , 


for x — 0, 1 .y = 0, 1.and#i and # 2 as given in ( 8 c) and m= 1, 2, ... Define 

\ i = a/(\ — d). K 2 * 6/(1 — d) y K 3 » c/(l - d) in (16). Since products of probability gen¬ 
erating functions correspond to convolutions of probability functions, an application of the tri¬ 
nomial expansion to (16) coupled with a termwise inversion of products of the form (17) with 
(15) gives 

THEOREM 1: The probability function of (16) with a, 6 . c. d > 0, a + 6 + c + d — 1, 
b + d < 1 , c + rf < 1 , is 

(19) f m (x.y)~ X —Btj x “ x f^i’/ra(x) *£ m (x y) */ ty (y). 

«.U y “PV- 

” I ~rr K > K f^ i t /xa(x-f)/ >y (y-T,)g B 1 (f.T > ). 

...a.? f-07,-0 

Jr 

where I runs over all a, /3. y > 0 such that a + (3 + y — m. The operator * for convolution 
over x is defined for two functions 6 i(x, y) and 6 2 (x, y) by 

61 * 6 2 - £ 6 i(£, y) 6 2 (x - ?. y). 

«-o 

The operator * is defined similarly. When /3 = 0 or y = 0 in (19), f x o(x) and / > 0 (y) ate taken 
to be unity. 

The distribution / m (x, y) will be henceforth referred to as the BNBia, b, c. p, q. m) dis¬ 
tribution. The distribution g„(x. y), only when v — m an integer, is a special case of / m (x. y); 
the distribution / m (x. y) is not in general infinitely divisible. The BNBia. b, c, p, q. m) dis¬ 
tribution is the discrete analogue to Paulson's (10) bivariate gamma distribution. 

As a direct consequence of Theorem 1 we obtain a closed form representation of Paulson 
and Uppuluri's ( 11 ) bivariate geometric distribution as 

COROLLARY 1: For m - 1, d < 1, the BNB(a, b, c, p. q, 1) distribution is 


NAVAL RESEARCH LOGISTICS QUARTERLY 


VOL. 28. NO 3. SEPTEMBER 1981 











BIVARIATE NEGATIVE BINOMIAL DISTRIBUTION 


365 


( 20 ) 


/.<* y) 


1 - d 


g i(x. y) + 


1 - d 


1 


1 + 0 , 

1 


1 + 0 


l - d l +e, l +0 


* g\(x, y ) 
*«i(jc, y). 


The requirement that the shape parameter m of f m (x, y) in (19) be integer valued limits 
the practical utility of the probability function. We are thus led to seek a representation for 
which the shape parameter may take on arbitrary real values. This is equivalent to first deter¬ 
mining whether f m (x, y) is infinitely divisible, and if so, to secondly determining an expression 
which is not tied to integer values m. In general, the BNB(a, ft, c. p, q, m ) distribution is not 
infinitely divisible (Block, Paulson, Kohberger [4]) and no such representation will exist. The 
BNB(a, b. 0 ,p, q, v ) and BNBia, 0. c, p, q. v) distributions are, however, infinitely divisible 
as are the BNB(a, 0. 0, p . q. v) distributions. 

THEOREM 2: The BNB(a. b, 0, p, q, v) and BNB(a, 0. c, p, q, v) distributions, 
b + d < 1 . c + d < 1, d < 1, v >0, are infinitely divisible. 


PROOF. In (16) replace m by v and write the resulting probability generating function 
when c = 0 as0„(c * 0). Then it follows that 


( 21 ) 


<Mc - 0) 


(1 - dW» $ 2 

v 

a 

1 - din 

1 - dib fli 2 


\-d 

l - (ft + d>in 


The term in square brackets is infinitely divisible. The term in braces is infinitely divisible if it 
is a probability generating function for arbitrary v > 0. But 


( 22 ) 


(1 - <tt|) 


1 - (b + d)«|/. 


exp 


|„|log| — 


1 - din 


(ft + d)<\> i 

explt' (log(l - din) - log(l - (ft + dbb i))l 


exp 


*> {If { 

\v T ((ft + d)'- d 0 — 
1 ,-i J 


1 + I I 


n - 1 
n — k 


ft*(ft + d) n ~ k ift j 


1 + £ ft(ft, rf, n. v)ift", 


say, on successive expansion of log(l — dib\), logd — (ft + dH\) and the exponential func¬ 
tion. For all v > 0, 0 < ft < 1, 0 < ft + d < 1, ft (ft, d. n, v) > 0. An application of 
Theorem 12.2.3 of Lukacs 18] yields the infinite divisibility of the probability generating func¬ 
tion (a (1 — dl> |)/((1 - d) (1 - (ft + dbb ]))}“■ Since <b„(c- 0> is a product of infinitely 
divisible probability generating functions, it, too, is infinitely divisible. 
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The constructive nature of Theorem 2 provides the representation we seek. The term H 
in (21) has probability function gjx. y). The term H 1 ' in (21) has probability function given 
by the termwise inverse of 

1 + X 6(6. A «. vVl* ")j. 

»-i 


1 - d 

Hence, the inverse of<6„(c — 0) of (21) is 


(23) 


/„(x, y\c - 0) 


1 - d 


g„(x, y) 


+ g„(x, y) * £ 6(6, d, n. v) (1 - p) n p x 

rt— I 


f x + n — 1 


We thus have proven 

COROLLARY 2: The BNBia. b. 0. p. q, v) distribution, b < l, b + d < 1, has proba¬ 
bility function f,.ix, y|c - 0) given by (23). 

Elementary computations provide the moments of the BNBia. b. c, p, q. v) family. In 
particular, the means and covariance matrix are provided in 

THEOREM 3. The mean vector and covariance matrix for the BNBia. b, c. p. q. v) dis¬ 
tribution are 


(24a) 

and 

(24b) 




1 = 


p \ 


ft) | 

P ) 


vti 2 


2 


v»,(l-M),) y(g</ 

<*\ <*\U 


' 1 1 - d 

o\o <r> 


V(C,d - ~ ^ V# 2 (l+# 2 ) 



1 - d 


where » | and « 2 are given in (8c) 

The conditional mean of Y given x is plotted against x in Figure 1. The data, taken from 
Bates and Neyman (31, as depicted in this figure, strongly suggests that a nonlinear regression 
function would be most suitable for describing the empirical relationship between X and Y. 
Unfortunately, the Bates-Neyman distribution does not admit of nonlinear regressions. The 
capability of allowing for nonlinear regressions seems to be fundamental in data analysis situa¬ 
tions. The regression function for BNBia, b. c, p, q. 1) (and, hence, also BNBia, b, 
c. p. q, m)) is nonlinear. 

THEOREM 4: The regression function £(EU) of the BNBia, b, c. p, q. 1) 
distribution, 6 * 0, is 


(25a) 


(25b) 


£( K|x) 


£( Tlx) 


_3— 

1 - q 


— *' +l . 6*0. 
6 [ m b J J 

1 c- 0. 

- dil - p) j 
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E[Y|x] 



o 10 20 X 

618 156 49 17 10 7 4 0 2 1 __ Number in mean 

276 88 33 10 8 3 1 3 0 


I'Kil'Rl. I. Theoretical and observed regression functions for Bates-Neyman Data 

where 

(26) m * p + (1 — p) (a + c). A = cm/[(o + c) (a + 6)] 
k - m/lm + 6(1 - p)l. <i “ 1 — a - b — c. 

PROOF: The z-transform of E( K|x) is defined by 

(27) z(£[T|x])- g(z)= Xz'*£mx]- £ z*'y/i(yU), 

x-0 x-0 

« ~ (I -f-tf,)l J ‘ 

(28) - (1 +«|) X E —— - y f,(x. y), 

x-0 y-0 M I 

since the marginal density of X is Pr(-T - x| - 
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But (28) may be expressed in terms of the probability generating function v) as 

A I 1 + 0 1 ) 

(29) *<*>-(i r^7 L - x l 


q 

1 


Az z(z - m) 

l - <7 

m + 6(1 - p) 

z-k (z-k)(z- 1) 


after some computation and simplification. The inversion of g(z) (Jury, (7|) gives (25a), an 
exponential regression function. Letting b — 0, c — 0 in (25a) gives (25b). 

The nonlinearity of regression is embodied in the term A >+l in (25a) with k defined in 
(26). The parameter b must be sizeable for small m in order that the regressions be nonlinear 
over a range of x not far removed from zero. Thus, bivariate negative binomial data for which 
this form of regression would be most applicable would possess appreciably more frequency 
counts along the x-axis than along the y-axis. 


The distribution g„ (x. y) is central in the discussions of all the probability distributions 
discussed to this point. The availability of an easily effected computational procedure for 
g,.(x. y) would facilitate the utilization of the negative binomial distributions. Since g,. (x. y) is 
expressible in terms of the hypergeometric functions, the Gaussian contiguous functions play a 
dominant role in any computational scheme. First, observe from the definition of 
F(a. b. c. z) that 


F(r + x, *>;♦>; z) — (1 - •z)~*'"\ 
F(y. v + y. v, t> \ z) = (1 — z) _ *"~ r . 
Thus, it follows immediately that 


(30a) 


(30b) 


g, (x. 0) ■ 
(0. y) 


(1 - d)( 1 - p)(l - q) 

* 

P 

X 

v + x - 1 

1 - z 


1 - z 


X 

(l — </)(!— p)( 1 - q) 

- 

<7 

>■ 

v + y — 1 

1 - z 


1 - z 


.V 


The use of relationship 15.2.18 of Abramowitz and Stegun [1] allows (15) to be written as 
j?„(x. y) = (1 - dY h\{x)h 2 {y) I ~~ ' ;~ + ~ [xF(v + x - 1. v + y\ p; z) 


+ (y + y) (l — z) F(y +• x, v + y + 1; v; r)]J. 

A few elementary manipulations give 

(31) y„(x. y + 1)» 7 ■ r ? ~- 7 l(x + y +v)g„(x. y) 

(y + 1 ) (1 — z) 

- piy + x - l)&,(x - 1 , y)l, 

which allows the entire probability distribution to be computed recursively from g„(x. 0) and 
g„( 0 . y), x ^ 0 , y ^ 0 . 
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Estimation of the parameters of these probability distributions generally require nonlinear 
optimization routines. Some of the computational burden can be reduced through judicious use 
of 


THEOREM 5: For the BNB(a. 0. 0, p, q.v) distribution and if v is known the likelihood 
equations for a random sample of size n are 


R = 0 


0 

where L is the ljkelihood function, x and y are the sample means for the marginal distributions, 
v + 1 ) S v (x v + 1) 

n„ is the number of observations for which X — x. 


(32a) 

91og£. 

da 

— +y - R = 0 
a 

(32b) 

Slog L 
dp 

(1 - p) - , - 

-— x + y — 

P 

(32c) 

dlogL 

dq 

(1 - q)y . - 
-+ y — i 

q 


R - — £ n x 


g,Ax. y) 


Y =» y, and g,Ax. y) is the probability function in (15). 


PROOF: If the probability function in (15) is differentiated with respect to the parame¬ 
ters a, p , and q the following differential-difference equations 

9g,Ax. y) 


(33a) 

(33b) 

(33c) 


da 

dg,Ax. y) 
9p 

dg,Ax. y) 
dq 


ad + d 


— + — 
P 1 - P 


+ T"— 
q 1 ~ q 


(x v)- -|- 
a 

y + 1 

q 

gjx, 

g,, (x, y) - 

1 

y + i) 

1 - p 

9 I 

g,, (a. y) - 

l 

y + 1 

l - q 

q 


gv (x y + 1), 


g,Ax. y + 1). 


result; these equations follow by using (15.2.1, Abramowitz and Stegun [1]) 

dF(a, b. c,:) ab r . . , , . . . . ^ 

---= — F(a + 1. b + 1, c + 1; z) 

or c 


and (exercise 1, page 296, Whittaker and Watson {13]), namely 

F(a, b + 1; c; r) — F(a. b\ r; r) = — F{a + 1, t + l;c+l;z). 

c 

The log likelihood function, log L , for a random sample of size n is £ n X} log g„ (x, y) 
and so 

d log L _ y 1 9g,Ax, y) 

da n " g,Ax. y) da 

Using (33a) and a few simple operations leads to (32a). Similarly, (32b) and (32c) are 
obtained. 
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From (32) it is clear that 

(34 ) 

P q a 

and these relationships are very useful in estimating the parameters via the method of max¬ 
imum likelihood. For v known or not, the conditions in (32) are necessary for a maximum 
likelihood solution to the likelihood function for b — c - 0. Therefore, (34) can be used to 
reduce the dimensionality of the unknown parameter space from four if v is unknown, to two 
by taking, say, p — qx/[qx + (1 - q)y] and a = vq/[( 1 - q)y\. We have used a nonlinear 
optimization computer program to solve for the parameter estimates. The dimensionality 
reduction permits extremely shorter running times. 

4. APPLICATIONS OF THE NEW BNB DISTRIBUTIONS 

This paper was originally motivated in part by the visual disagreeableness of the fit of the 
regression function 

g(F|s)- *±±*1 

1 - q 

of the Guldberg-Bates-Neyman distribution of (5) to the observed data as depicted in Figure 1. 
The maximum likelihood fit of the Guldberg-Bates-Neyman model to this data is dismal as 
measured by a \ 2 goodness-of-fit, indeed, a linear regression is clearly not appropriate for this 
data. We fitted the BNB (a. b. c.p. q. 1) and the BNB (a, b. 0. p. q. v) to these data 
(Bates-Neyman, 13]) but obtained b - c -* 0 in the first case and b =* 0 in the latter. Even 
though the degree of fit as measured by \ 2 increased substantially, the overall fit was still very 
poor. The conclusion that the Bates-Neyman data possesses characteristics which preclude the 
possibility of a good representation by a bivariate negative binomial seems inescapable. There 
is thus no reason to present any of our results concerning this data. 


TABLE 1 — Bivariate Data Sets 


Description 
of Data 

Sample Size 
and 

Correlation 1 

Marginals: 

Mean and Variance 

Parameters 2 of 
Univariate 
Negative 
Binomial 
(EQ. (D) (v. 0) 

Fig of Uni¬ 
variate Nega¬ 
tive Binomial 
to Marginals 
(X 2 , df. P) 3 

Arbous-Sichel 

Absenteeism 

248, 0.73 

(X) 1947 

(Y) 1948 

4.70.18.66 

4.48.18.66 

1.58,2.99 

1.56,2.88 

8.9, 13,0.78 
11.0,12,0.53 

Bates-Neyman 

Diseases 

1286, 0.42 

(X) Dig. D. 

(Y) Res. D. 

1.40, 5.06 
5.32,22.13 

0.53,2.62 

1.69,3.16 

12.3,10,0.26 

25.4,20,0.20 

Aircraft 

Flight Aborts 

109,-0.16 

(X) lst 6 mos 

(Y) 2nd 6 mos 

0.62, 1.03 ' 
0.72, 1.08 

0.95,0.66 

1.51,0.48 

0.24,1,0.62 
6.3, 1,0.01 


'Pearson product-momem 2 Via method of moments 2 Probability of exceeding computed V 2 value with indi¬ 
cated degrees of freedom 


Table 1 shows selected summary results for ail the data examined in detail. The first 
column identifies the data, column 2 provides the sample size and correlation, column 3 
specifies the marginal random variables and shows the associated sample means and variances. 
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columns 4 and S show the parameters of the univariate negative binomial fitted to the marginals 
and the associated x 2 values, degrees of freedom and probability levels, respectively. Results 
from this table will be presented along with a discussion of the individual data sets. The absen¬ 
teeism data of Arbous-Sichel [2] is examined first. 

From Table 1 we see that the marginal distributions are fitted rather nicely by the univari¬ 
ate negative binomial. Coupled with the fairly large sample correlation coefficient, it seems rea¬ 
sonable to expect a bnb distribution to adequately describe the data. Arbous and Sichel fit the 
G - B - Nil, 9, v) model to their data and report a x 2 of 17.0 on 13 degrees of freedom 
(P - 0.20), indicating reasonable agreement of the data with the model. We give in Table 2 
the expected cell frequencies obtained from the BNB (a, 0, 0. p, q, v) distribution along with 
the observed cell frequencies for all but 12 of the nonzero observed cells. The expected cell 
frequencies for the two models are about the same and we thus expect a similar probability P to 
obtain for the fit of the BNB (a, 0, 0, p, q,v). Since Arbous and Sichel do not show their 
grouping, we have not computed the x 2 statistic. The fits are comparable as judged by cell resi¬ 
duals. 


Although the fit of the G - B - Nil, 9, v) model to the observed data is reasonably 
good, the authors point out that 12 of the 18 observed means lie below the theoretical regres¬ 
sion function £"( K|.v) — qiv + x)/i\ - q). The BNBia. 0. 0. p, q. v) model, via maximum 
likelihood estimates, gives rise to a regression function for which only 10 of the 18 observed 
means are less than the predicted values. 

An attempt to fit the BNBia, b, c. p, q, 1) and BNBia. b. 0, p, q, v) models to these 
data was made but we obtained maximum likelihood estimates b - c - 0 in the first case and 
b — 0 in the second. The lack of influence of the parameters indicates the apparent adequacy 
of linear regressions for the data. Although the fit of BNBia, 0, 0, p, q. v) is adequate, a 
bivariate beta binomial is a more appropriate distribution for this data. 

Table 3 provides observed and expected cell frequencies for flight aborts for 109 aircraft. 
The fundamental data consist of 109 pairs of observations (x,, y,), where x, represents the 
number of aborts by aircraft i in the first 6 months and y, the number of aborts by aircraft / in 
the 2nd 6 months of a one year period. Most of the data fall in the Or, 0) and (0, y) cells and 
this implies that the correlation should be negative. Note that the probability contours are 
unlike those traditionally associated with negative correlation and that the distribution has its 
probability concentrated along the two axes. The maximum likelihood estimates for the 
BNBia, b, c, p, q, 1) model are a - 0, b - .68, c — .32, p - .17. q - .33. These values pro¬ 
duce an estimate of correlation of - .13. The solid borders in Table 3 indicate the grouping we 
have used for x 2 - We find the x 2 statistic to be 10.2 on 6 degrees of freedom or F* 0.12, a 
marginally acceptable fit. The observed x + y has frequencies 34, 37, 17, 10, 9, 2 for 

x + y — 0, 1.5, respectively, and 0 for all x + y > 5. The mode is at x + y — 1 which 

makes more plausible a BNBia. b, c. p. q, v) model with v > 1 since with v ■ 1, the mode is 
at x + y - 0. Unfortunately, the BNBia. b, c, p. q. 1) is not infinitely divisible (Block, H.W., 
A. S. Paulson and R.C. Kohberger 14}) and hence we will not be able to produce a better fit to 
this data in the class of models introduced herein. One might reasonably be hesitant to call a 
distribution which is not infinitely divisible a "true" negative binomial. 
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TABLE 3 — Observed and Expected Cell Frequencies 
of Flight Aborts for 109 Aircraft 



1st Six Months 

'Bivariate negative binomial distribution of (20). Maximum likeli¬ 
hood parameter estimates are: a - 0. 6- 0.6820. i -0.3179. 

0 1655. q- 0 3299 There results \ 2 - 10 2 and P - 0.12 for 
<!t - 6 . 


5. DISCUSSION 

We have produced and examined properties of a class of negative binomial distributions 
which admit of nonlinear regressions and negative correlations. The most useful subset of this 
class is infinitely divisible and therefore allows for a flexible fit to data. A nonlinear optimiza¬ 
tion routine was used to produce the maximum likelihood parameter estimates. 

Even though the development of models which admitted of nonlinear regressions and 
negative correlations provided some of the impetus to this work, it soon became clear that 
bivariate models with a different shape parameter for each margin would be eminently useful in 
the modeling of bivariate data. A large number of attempts at developing such appropriate 
negative binomial distributions of this type met with only limited success. It appears that a 
model of a character much different from those heretofore discussed in the literature is 
required to produce a multivariate negative binomial distribution which is simultaneously 
infinitely divisible, allows for nonlinear regressions, and different marginal shape parameters. 
The comparison of adequacy of agreement between data and model was based on the usual x 2 
goodness-of-fit statistic, a procedure with serious shortcomings. An alternative procedure for 
multivariate goodness-of-fit would be a welcome addition to the multivariate literature. 
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The most appealing properties of our BNBia, b, 0, p, q, v) and BNB(a. 0, c, p, q, v) 
distributions are their infinite divisibility and nonlinear regressions. However, unless nonlinear 
regressions are called for, it appears that any of the bivariate negative binomials would produce, 
via maximum likelihood, an acceptable match to bivariate negative binomial data. It should be 
emphasized that the maximum likelihood estimates need not produce the best fit of a model to 
the data as measured, for example, by x 2 . Further, under maximum likelihood and ordinary 
least squares, each observation is weighted equally; if the number of "taif observations relative 
to the complete frequency count is small, then these procedures need not produce a visibly 
nonlinear regression even when one is clearly appropriate. In this case an "anti-robust" pro¬ 
cedure may be preferable to maximum likelihood and least squares in order that legitimate out¬ 
lying observations be weighted more heavily. A case in point is provided in Figure 1 where the 
bivariate negative binomial model is not appropriate but the data are correct and, unfortunately, 
the model has been given preference. 

Extension of our results to dimensionality in excess of two would require a little care in 
ensuring the simultaneous infinite divisibility and nonlinear regressions. The computational 
problems would increase with the complexity of the model which implies a necessity for parsi¬ 
mony vis-a-vis realism tradeoff. 

Finally, we wish to emphasize that all of our results and discussion has a dual counterpart 
in the bivariate gamma distributions of Paulson [101. Accordingly we would define and work 
with bivariate gamma distributions BVG(a. b. 0. p. q. v) and BVGla, 0, c. p, q. v) in com¬ 
plete analogy with the preceeding results. 
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ABSTRACT 

A model for proper scheduling of inspections is considered, if system 
failures can be detected only by checking. Two cases are analyzed: replacement 
and no replacement of a failed system. On condition that no or only partial in¬ 
formation on the lifetime distribution of the system is available, minimax in¬ 
spection strategies are obtained with respect to cost criterions. 


1. INTRODUCTION 

The following situation is considered: At time / *= 0 a system starts working. The time to 
its failure (lifetime) is a random variable X with the probability distribution function Fit), 
Fi+0) -0. A system failure is assumed to be known only by inspecting. Each inspection 
entails a fixed cost c and takes only negligible time. On the other hand, a downtime t of the 
system (= time between system failure and its detection) gives rise to cost v(/), where v(/) is 
a continuous, strictly increasing function of /, f > 0; v(0) = 0. 

Let be S - {rj an inspection strategy (at time t k the Ath inspection takes place, when no 
failure has been detected before, 0 - t 0 < h < h < •••)• The aim of this paper consists in 
deriving inspection strategies, which are optimal with respect to some cost criterions. Results 
are obtained in the cases where the lifetime distribution is completely or partially unknown. In 
the latter case, the expected system lifetime is assumed to be known. 

To the knowledge of the author, the model described has been first analyzed by Derman 
[10] (unknown lifetime distribution) and Barlow, Hunter and Proschan [2] (known lifetime dis¬ 
tribution). Modifications of the basic model described have been treated e.g., in [I, 3, 12, 14, 
15]. This paper summarizes and extends results obtained by the author 14, 5, 6, 8]. 

2. INSPECTION WITHOUT REPLACEMENT 

If an inspection strategy (short: strategy) S - Jr*} is in effect, then the expected total) 
loss cost K iS, F) up to detection of a system failure amounts to 

(1) K(S, F) - £ /'**' g k (t, t k +\)dFit), 
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where 

g k (x. y) - (k + l)c| + v(y - x), 0 < x ^ y. 

If a downtime cost can arise only in the finite interval ( O, D, then inspections after T are not 
necessary. Applying in this case a strategy S„ - {/ k | with 0— t 0 < r ( < ... < t„ < /„+, - T 
the expected loss cost is 

(2) K(S n . F) - £ ' & (/. i k+> )dF(t) + (n + l)c,(l - F(D). 

*-o * 

Next F(t) is assumed to be entirely unknown. This assumption leads to the cost criterion 

(3) K(S) = sup K(S, F). 

F€F 

where Fis the set of all probability distribution functions F(r) with F(+0) = 0 and K(S. F) is 
given by (2). A straightforward estimation of K (S„, F) yields 

K(S„)~ max &(/*, r* +1 ). 

A-0. (. n 

A strategy, S*., satisfying 

K(S* n -) - min K(S) 

s 

is called "minimax strategy." 

THEOREM 1: Let n* be the largest integer n satisfying 

(4) T v-'ac,) < T, 

where v*'(jr) is the inverse function of v(f). Then there exists a unique solution S*. - {/•}, 
0 < r* </*<...< /J. < 7, of 

(5) g 0 W. f|)-g|(/|. r 2 ) = T) 

and S*. is minimax strategy. 

PROOF: With n — n* Equation (5) is equivalent to 

(6) v(8*) - v(8 0 ) - kc\. k «■ 0, 1. rt\ 

8* >■ T, & k * f/j+i — t k , k — 0. 1 , n. 

k -0 

Thus, in view of the definition of n *, v(/f) ^ [n* + 1)c ( holds, and for any strategy S r with 
r - «• + 1 

X(S;.) - K{S r ) < v<rp + c, - (r + l)c, < v(f?) - (n* + l)c, < 0. 
Therefore, a strategy S r with r > n* cannot be minimax strategy. 

The functions g k (x, y), k — 0, 1.have two obvious properties: 

(i) g*(jc, y) is strictly increasing in y, if x is fixed, 0 < x < y. 

(ii) g*(jr, y) is strictly decreasing in x, if y is fixed, 0 < x < y. 
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Let — {c^} the solution of (6) with 0 < n < n*. Assume there exists a strategy S n - {/*} 
satisfying 

(iii) K(S n ) < K(S'„). 

Let (, be the smallest of the inspection times t k with the property i, & /,'. Because of (iii), 
< K(S'„). Taking into account (i) this implies /, < Analogously, (i) and (ii) 

yield r l+1 < f,'+i (for otherwise (iii) could not be satisfied). Continuing in this way, one finally 
obtains t„ < and, therefore, g(t„, T) > K(SJ) (see [16]). Hence, there cannot exist a stra¬ 
tegy S„ satisfying (iii). If and 0 ^ rt\ < n 2 < »*, are solutions of (6) with n = n t and 

n - w 2 , respectively, then it can be proved analogously that K(S' n ,) < Af ($,'). This completes 
the proof of the theorem. 

COROLLARY: n* is the largest integer n for which there exists a strategy S' — {r*| satis¬ 
fying (6), 0 < f( < < ... < t'„ < r,',+i =• T. Similarly, r* is the smallest time point for the 

first inspection, which can belong to a solution of (6). 

An analogous property of the first optimum inspection moment with respect to K(S, F). 
if /(/) - F'(t) is known and of type PD 2 (polya density function of order 2), has been shown 
in [5J. 


Derman [10] has already obtained the minimax strategy in case of v(r) = c 2 f, c 2 > 0. It 
is given by 


n* + 1 2c- 


(n* — A' + 1) 


, k * 1. 2. n* 


where n* is the largest integer satisfying 


n(n + 1) < 


2 CjT 
c I 


Now suppose the lifetime distribution of the system to be unknown with the exception of 
the expected lifetime n — E(X), 0 < n < °°. The condition that downtime cost occurs only 
in (0, T) is now dropped. Let 


K^iS) = sup K(S, F). 


where K (5, F) is given by (I) and 


F. F € F. M - f” t d Fit) 


A strategy, S*, satisfying 

K^iS;) - min K^iS) 

is called "partial minimax strategy." 


Within a more general model Hoeffding [11] has shown that for any S - |r*} 
sup KiS, F) *- sup K(S, F), 
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where F™ is the subset of those elements of having exactly two points of increase. Routine 
calculations yield 

(7) KAS)- sup GAS), 

m< j 


where 


<V S) ” 7—~ fiU* ',+1) + ; — *>((,-. r >+ i) 

V 'i 'j '/ 


and m — w (S) is defined by 

( 8 ) t„, < n < f m+ , (see [81 for details). 

A strategy, 5 - { 4 }, is called strictly periodic with the inspection interval 8 , if 
8 - 8 * - 4 +i - 4 for all /c « 0, 1, ... ; notation: S - S mi . K^S) has a simple structure for 
strictly periodic strategies: 

(9) A:„(S (8 >) “ Y F- + v( 8 ) + c,. 

This results also from a straightforward estimation of K (S'® 1 , F) if F € F M : 

.(* + 1)8 
'*8 


A(S (S \ f) = T f K* + lk, + v((* + 1)8 - r)l dFU) 
£o Jk * 

“ /• (* +1 >8 

l(* + l)c, + v(8)|tf(f) 


£ U 8 ) [FAk + 1)8) - F(k8)] + v( 8 ) + c, 


^ — fi + v( 6 ) 4* c |. 
o 

Thus, if v(r) is differentiable the optimum inspection interval 8 * with respect to Af M (S ,S) ) 
satisfies 


( 10 ) 


8 2 v'( 8 ) — fic ,. 


There exists always a positive solution of this equation. Especially, for v(r) — c 2 t the unique 
solution is 

( 11 ) 8 * “ -Jnc\/c 2 . 


THEOREM 2: There exists a strictly periodic partial minimax strategy. 

PROOF: Assuming there exists a strategy S - { 4 } with the property 
( 12 ) K^S) < K^S^). 

G,j(S) can be written in the form 

G,j(S) — (/ + l)c, + v( 8 ,) + (fi — t,)c 1 Qjj 
(J - /)C| + v( 8 j) - v( 8 ,) 
a " " Uj ~ Oc, 


with 


■, 0 </<«</ 
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Further, let a, - sup a„ and a = min a,. Without loss of generality it can be assumed 

)>m J OsGSJm 

0 < a < Putting 8 - \la there holds in view of (9) and (12) for all 0 < i < m 

(j 4- 1 >C| + v( 8 ,) 4- 0* — t,)c|fl < M c i a + V |~J + c «- 

Starting with / — 0 induction yields 

(13) 8, < 1 /a, 0 < / < m. 

Let be /' 0 defined by a, u = a. Then it must be 

(j ~ to )c i + v(Sj) - (S, (i ) 

- - -T- < a, j > m. 

“ '.„) fi 

Using (13) it can be inductively seen that 8 , < 8 I(1 , j > m. But this implies 
a = sup a ^ 1/8, u , contradictory to (13). Hence, there cannot exist a strategy S satisfying 
( 12 ) and the theorem is proved. 

The proof given shows, moreover, that every partial minimax strategy must be strictly 
periodic. Its inspection interval is a solution of Equation (10). 

Analogous problems, where, instead of the expected lifetime, a percentile of the lifetime 
distribution is assumed to be known, have been treated in 15, 8 , 13, 15]. 

3. INSPECTION AND REPLACEMENT 

In this section an obvious extension of the basic model with fj. known is considered. 
Immediately on discovery of a failure the system is replaced by a statistically equivalent one. 
Each replacement requires a fixed cost c, and a fixed time d. The inspection-replacement pro¬ 
cess is continued unlimitedly. 


In what follows it is assumed v(r) — c 2 t, where c 2 is restricted by 
(14) (c, + c,)/#x < c 2 . 

Otherwise, the expected loss cost per unit downtime of the system would be smaller or equal 
than the mean loss per unit time which arises by "ideal inspection and replacement" (i.e., failure 
of the system is detected immediately and replacement takes negligible time). But then inspec¬ 
tion and replacement are uneconomical from the first. 


The expected length of a cycle (time between two neighboring replacements) by applica¬ 
tion of the strategy S — {r fc ] is 

L(S, F) - £ /* +1 lF(/* +t ) - F(t k )] + d. 

*-o 

Thus, the expected loss cost per unit time C(S, F) amounts to 
K(S, F) + c 3 


C(S, F) 


L(S, F) 


where K(S, F) is given by (1). 
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Brender 19] has dealt with the problem of calculating an optimum inspection strategy with 
respect to CiS, F) if F is known. If F is completely unknown the corresponding mimimax 
inspection strategy is easily seen to be trivial—no inspection and replacement at all (a nontrivial 
minimax inspection strategy exists if v(r) is strictly convex [7]). 

Let 

C„(S) = sup CiS. F). 

To get the corresponding partial minimax strategy, the following lemma is needed. For being 
only a slight generalization of an approach used by Brender (9] its straightforward proof can be 
omitted (see ( 6 ] for details). 

LEMMA: Let be 

QiS, x) = sup ltf(S, F) + c, - xL(S, F)] 

and assume for all x, 0 < x < c 2 , the existence of a unique strategy Six) and of a number x () , 
0 < x„ < c 3 , so that 

QiSix), x) = min QiS, x) and £>(S(x ( )), x 0 ) = 0. 

Then Six () ) is the unique partial minimax strategy and jr<> = C^iSii r ft )). 

KiS, F) - xLiS, F) has for every x, 0 < x < c 2 , the same functional structure as 
K(S, F). Hence, Theorem 2 secures the existence of a strictly periodic strategy S(x). 
According to (11) its inspection interval Six) is given by 

(15) Six) = - N //xc,/(c 2 - x). 

It follows from (9) and (15) 

QiSix), x) — 2 -y/ftciicj - x) - xifi + d) + c , + c 2 d + c,. 

Of course, QiS(O), 0) > 0, and in virtue of (14) Q(S(c 2 ), < 0. Hence, a unique solu¬ 

tion x ( i of @(S(x), x) = 0 exists: 



Minimax inspection problems for replaceable systems, where, instead of the expected life¬ 
time, a percentile of the lifetime distribution is known, have been treated in (6, 16]. 
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THE ESTIMATION OF PiX < Y) FOR DISTRIBUTIONS 
USEFUL IN LIFE TESTING* 


A. P. Basu 

University of Missouri 
Columbia. Missouri 

ABSTRACT 

In ibis paper ihe reliability function R = Pi V < Yi has been estimated 
when V and Y follow gamma, exponential or bivariate exponential distributions 
The paper is partly expository. 


I. INTRODUCTION 

Let X and Y be two random variables with cumulative distribution functions Fix ) and 
G(y) respectively. Suppose Y is the strength of a component subject to a stress X. Then the 
component fails if at any moment the applied stress (or load) is greater than its strength. The 
stress is a function of the environment to which the component is subjected. Strength depends 
on material properties, manufacturing procedures, and so on. The reliability of a component is 
the probability that its strength exceeds the stress. From practical considerations it is desirable 
to draw inference about the reliability function. 

The above model was first considered by Birnbaum 121 and has since found an increasing 
number of applications in many different areas, especially in the structural and aircraft indus¬ 
tries. For a bibliography of available results see Basu [1 ]. 

In many situations, the distribution of X (or of both X and Y) will be completely known 
except possibly for a few unknown parameters and it is desired to obtain parametric solutions. 
Thus, in case of missile (lights, the stress may be expensive to sample, but the physical charac¬ 
teristics of the missile system, such as the propulsive force, angle of elevation, changes in 
atmospheric condition, and so on, may all have known distributions; consequently, the distribu¬ 
tion of stresses can be calculated. Church and Harris 14], Owen, Craswell and Hanson 18], and 
Govindarajulu [6] have considered the above problem under the assumption that X and Y have 
normal distributions. Since in many physical situations, especially in reliability, exponential and 
other distributions provide more realistic models, it is desirable to obtain estimators of R for 
distributions useful in life testing. In section 2 we consider gamma and exponential distribu¬ 
tions under the assumption that X and Y are independently distributed. The case of a bivariate 
exponential distribution is studied in section 3. Two distribution free procedures are mentioned 
in section 4. The effect of misspecifying the model is considered in section S and a numerical 
example is given in section 6. 


' fhis research was supported in pari under eonir.icl N00O14-78-C-0h55 for ihe Office of Naval Research 
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2. (LAMM A AND EXPONENTIAL DISTRIBUTION 


(2 It 
( 2.21 


Let A and ) be independently distributed with density functions 
/ (v) = e " V .v > 0, p > 0 

I \p ) 

ft < i ) = c ,u v“ '. v > 0. q > 0 

1 (q ) 


respectively. Then 

(2.3) ft„„ = P(X < 



V n P + k) a'jB* 

r<p>r<* + i) ( a +/3)'’* k ' 


Here pand q are assumed to be known integers. 


Note: /?n = «/(u + (i), and /?,,, = ftfi for all />. Also, 

i = y i n - 

* II P H -0 

In particular, 

ft.’i = ftfj. Rx | = ftj'|, 

ft>: = ft,i + (j3/tv)/?u 

«,,= ft,, + 03/a)ftf, + (0/«)ftj,. 

ft;: = ft, 2 , (3 - 2ft,,). 

If ft u is close enough to one, as is expected for items with high reliability, 

ft,„ s j (I - ft,.)- 

Expressions for ft,,, and ft|„ indicate that in this case the expression for the reliability is not 
strongly dependent on the choice of the parameter p (especially if p is small) and the distribu¬ 
tion of X can be approximated by the exponential distribution without much distortion in the 
value of ft,,,. However, so far as the parameter q is concerned, the situation is quite reversed. 
The value of reliability is heavily dependent on the choice of the underlying distribution of Y 
and one has to choose the value of q more carefully. Later in section 5, we shall further study 
the effect of misspecifying the parameters p and q and confirm our conclusions by numerical 
studies. 


If two independent random samples, (A*,, X 2 . X,„) and (K,. Y 2 . K„), from the 

two gamma populations are available, maximum likelihood estimators (m.I.e.) of a and (i are 

given by « = -4 and= ~=. Hence, m.I.e. of ft,,,, is 


(2.4) 


o - V I (ft + k) a r f} 1 ' 

h r(ft)r(fc + i) («+j§)^‘' 


As special cases, if q = 1 , that is, if X follows the gamma distribution and Y follows the 
exponential distribution. 
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Finally, if both p and q are equal to l, we have the case of two independent exponential distri¬ 
butions, and we have 


Since R in 2.6 is a function of —, the exact distribution R can be obtained in the exponential 

X a 

case. It is well known that — — follows the ^distribution with (2m, 2n) degrees of freedom. 
. Y 0 

Thus, the distribution of 7?n follows. The result will be used later in section 5 to compare the 
performance of independent exponentials with those of dependent exponential models. Using a 
theorem in Rao [10], Theorem 6a.2, page 321, the distribution of R pi4 in each of the above 
cases, for large m and n, can be shown to be normal and hence an estimate of the asymptotic 
confidence interval for R can be obtained. Thus, in this case with m = n we have for large n, 
\fn (R pq — R nu ) ~ N(0,cr pq ). Expressions for R p<i , R nq and c r 2 r for a few selected values of p 
and q are given below. 


Y+l' <r ' ,= 

(X -+- 2 K) 2 


2 a ~0 2 
(a + / 8) 4 

21 (a+ 0 >‘ 


R _ « + _ 

12 ct + fi (a + 0 ) ! 


Y 2XY 

(Y + 2X) (Y + 2X) 2 


= 6 a 4 
(a+0) 6 


R = + 2a ^ R = ^ 

“ (a +/3) 2 (a+(i) } ‘ 22 (X + Y) 2 


+ (AT + K) 3 ' 


<t 22 = 36(a0) 4 /(a + 0) 4 


3. BIVARIATE EXPONENTIAL DISTRIBUTION 


Since the exponential distribution is considered a useful model in life testing problems, it 
is desirable to consider bivariate analogues of univariate exponential distributions which will 
have properties similar to those of the univariate exponential distribution. Marshall and Olkin 
[7] have proposed a very important bivariate exponential distribution (BVE), which is given by 


F(x, .v) = P(X > x. Y > v) = e 


X-K fl -A | ylTlilxf V.l ) 


0 < X,. X,. X |2 < oo, X, +x ]2 > 0, X 2 +X,J > 0(x > 0, y > 0). 

The BVE arises in several natural ways and is considered a useful model in reliability with 
appealing properties. 

Let (AT,. Y,), /” 1, 2. n be a random sample from (3.1). We shall estimate 

P(X < Y) when (X, Y) follows the BVE. It can be readily obtained from (3.1) that 


R = P(X < Y) = 


X i 4- X i + X i 
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Hence, R is estimated by 

(3.3) R - -—— 

A | + A 2 + A |2 

where A), A 2 , and X 12 are the maximum likelihood estimators of A t , A 2 , and A l2 , respectively. 
Various authors have considered maximum likelihood estimation of the parameters A,, A 2 , A l2 . 
However, no explicit solutions of these parameters are available. In order to obtain an explicit 
form for R, we replace the m.l.e. by some special ad hoc estimators called the "1NT estimators 
of Proschan and Sullo [9] which have very high asymptotic relative efficiency compared with 
the m.l.e. estimators. Let n, - number of pairs such that X ( < Y h w 2 - number of pairs with 
X, > Y, and n 0 — numbers of pairs with X, - Y r 


The "INT" estimators are given by 


(3.4) 

«i n 

A. | “ 


«l + "0 £ x 

f-t 

and 

An” «n \\ + -— 


n2 

n 2 + n 0 


[ /i| + n Q n 2 + n 0 ] 

Proschan and Sullo also prove the following theorem. 


£ max(T,. P,). 


THEOREM (Proschan and Sullo): n 1/2 ( A„ - A) is asympoto, sally trivariate normal with 
mean 0 and dispersion matrix I - (<r, y ) where, using the suffix n to denote dependence on the 
sample size, A„ = (A b A 2 , A 12 )\ A = (A b A 2 , A 12 )', 0 = (0,0,0)' 

(3.5) o’ 11 “ (A,(A — A ,A 2 yf ’), 
v\ 2 ~ A, 2 A 2 2 A 12 (Ay,y 2 ) _l , 

<t 13 — — A !A jA 12(A 1A 2 y 2 2 + Ay \ l )/(OoV |)» 

o'22 ” X 2 (X — A |A 2 y 2 '), 

cr 23 = — A |A jA 1 2 (A |A 2 y 1 2 + Ay 2 ] )/(Ooy 2 ), 

O’M" A j 2 (A - k(0Q 2 (k \y 2 } + A 2 y f 3 + 2.‘-,A 2 y f 2 y 2 “ 2 )J, 
and 

A “A| + A 2 +A| 2 , y 1 ” A 1 + A| 2 , 

y 2 " x 2 + x | 2 

#o — E[max(A', K)1 - yj -1 + y 2 _l - A H . 

R is then estimated by /? - A J(A , + A 2 + A )2 ) rather than by R. Since R is a totally 
differential function of A,, A 2 , and A, 2 by Theorem 6a.2(ii), page 321 of Rao [10], >/n (R - R) 
is asympototically normally distributed with mean zero and variance cr 2 , where 

,, ,, 2 W , W , . , , * iV 2 , , , . , W 

(3.6) o- 2 - — <7,, + — (tr 22 + 0 - 33 ) - 2 -j-j- (O’ 12 +O’|.r) + 2 ^f(T;j. 
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Thus, from (3.3), (3.4) and (3.6), one-sided and two-sided confidence limits for R can 
readily be obtained. 

To check the adequacy of the large sample approximation when the underlying distribu¬ 
tion is BVE, computer simulations were made. For various values of (X t , X 2 , X, 2 ) $00 sets of 
random samples of size n (n ** 10, IS, 20) from bivariate exponential distributions were 
obtained and the empirical distribution of R was obtained. Even for sample size as low as 10 
the exact distribution is found to be well approximated by the normal approximation The 
situation improves as the sample size increases. 

4. DISTRIBUTION FREE PROCEDURES 

In all the cases considered in the previous section to check adequacy of normal approxi¬ 
mation, the value of R is rather small (R < .7$), whereas applications of interest would be for 
systems with high reliability (/? > .90). Unfortunately, in all the cases with R > .90 con¬ 
sidered, the sample estimate of the variance of R came out to the negative or very close to 
zero. (The situation is similar to the problem of having ’negative* estimator of variances in 
analysis of variance problems.) 

To study the cases for which R > .90, we therefore consider the following estimators of 
R: 


For bivariate data, a rather natural way to estimate R — P(X < K) would be based on 
the binomial distribution. Let 7" be the number of cases for which X < Y. Then 7" is a bino¬ 
mial random variable with mean R and variance nR(\ - R). We therefore can obtain an exact 
binomial confidence interval or an approximation two-sided 100 (1 - y)% confidence interval 
given by 

(4.1) jy? - Z y ~ * 2 , R + Z y y^klLzAl , 

where R - Tin and Z y is such that <t>(Z y ) - (I - y/2), where 0( ) is the standard normal 
distribution. 

If X and Y are assumed independent, a second estimator is the following nonparametric 
confidence interval proposed by Govindarajulu ( 6 ] and is based on the Wilcoxon-Mann- 
Whitney statistic. 

Let (Af|, X 2 . X m ) and (K,. Y 2 . Y„ ) be two independent samples of measure¬ 

ments from populations with distribution functions F(x) and G(y) respectively. Let 

jl if X , < Y, 
ifi(X,. Y,) - |q otherwise 

m n 

then U • £ H'M-T,. T,) is the well-known two sample Mann-Whitney statistic. That is, 

i-i /-i 

U - number of pairs ( XY ,) such that X, < Y r Govindarajulu 16] has explicitly derived 
one-sided and two-sided distribution free confidence bounds for R (actually Govindartyulu 
derived confidence bounds for 1-/0 based on the asymptotic normality of R - U/mn. In 
particular, for the two-sided case, Govindarajulu showed that for all Fand G and large m or n , 
the solution e y of the inequality 
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IKK 

,4 ’< /»<|R - R\ < « y > > 1 - y. 0 < y < I 

in gti' cn b> 

«, > (4*>) 1 •’<!> '(I - y/2). 

litre »• - minim, n) and <t>( ) is the cumulative distribution function of standard normal distri¬ 
bution In particular, if m - n. a 100 (1 — y)% confidence interval is given by 

(4 it (R - « v . R + « y ). 

• here R and are as defined before, and 

A 

<v ” 2\fn 


5. EFFECT OF MISSPECIFY1NG THE MODEL 

In section 2 it was pointed out that for value of P(X < Y) close to unity, R w is less sen¬ 
sitive to the variation of p and varies considerably for varying values of q. To study the effect 
of misspecifying the model we carry out the following Monte Carlo experiment. 

Let G(u. p) and G(/ 3. q) be two given gamma populations with known parameters 
<«, p) and 03. q) respectively, where o * 19 and/3 — 1. In this case, if p “ q — 1, R n = .95. 
By choosing different pairs, (p. </), we get different pairs of gamma distributions. 

Let (Y|, X 2 . X„) and (K,. Y 2 . Y„) be two random samples from G(a, p|) and 

G(/S. q i). Since p\ and q i, the true values are not known, there is a possibility that we will 
choose a different pair of distributions as the true model. Let us assume that the above sam¬ 
ples have come from populations with distributions G(a, p 2 ) and Gift, q 2 ). Thus, we would 
estimate R n , q , instead of estimating the true value R,, „ , and compute a confidence interval 
based on R,, w> . We would not commit much specification error if this confidence interval con¬ 
tains the true unknown value For a given n and two pairs of values (p ( , q t ) and 

(p 2 , q 2 ) the above procedure is repeated 1000 times and a count is made of how many times 
the true value R,,^ is contained in the confidence interval based on R P}II} ■ We repeat the pro¬ 
cedure for different values of n and different combinations of (p\, q |) and (p 2 , q 2 )- The 
results are given in Table 1. Here n is chosen to be 5, 10 and 25. All combinations of the fol¬ 
lowing pairs of values are chosen for (p t , q t ) and (p 2 , q 2 ): (I, 1), (1,2), (2, 1) and (2, 2). 

From Table 1 we can make a number of conclusions. First, note that no parametric 
method performs well in all situations. For all the gamma models considered, the procedure, as 
anticipated in section 2, is robust for small variation in p. However, it is sensitive to variation 
in q. The nonparametric confidence intervals based on the Wilcoxon-Mann-Whitney statistic 
performs well in all cases. However, in each case the width of the confidence interval for an 
assumed model is too large compared with the corresponding width based on the parametric 
models. In each case the parametric method is to be preferred, especially if the parameter q is 
reasonably well specified. 
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TABLE 1 — c. Number of Counts for N — 25 


ASSUMED MODEL 


TRUE MODEL 

CO. 1)= G(2, 1) GO. 2) C(2.2) 

Exponential 


.95 

.9025 

.9975 

.99275 

GO. 1) 
exact 

945 

964 

0 

0 

GO. 1) 

normal 

approx. 

934 

964 

0 

0 

G( 2. 1) 

768 

812 

0 

0 

GO. 2) 

6 

82 

925 

982 

G(2. 2) 

l 

27 

754 

928 

*NP pr 

1000 

998 

1000 

999 


'Nonparameiric Procedure 


6. AN EXAMPLE 

To illustrate the computation of confidence intervals let us consider the following exam¬ 
ple. Fifteen items of random strengths K t . K 15 are subject to random stresses 

X\, X 2 , ... T| 5 . To estimate the reliability function P(X < Y) random samples of 15 pairs 
of (T, Y) values are drawn and given below. 

Pair No. 


1 

.0352 

1.7700 

2 

.0397 

.9457 

3 

.0677 

1.8985 

4 

.0233 

2.6121 

5 

.0873 

1.0929 

6 

.1156 

.0362 

7 

.0286 

1.0615 

8 

.0200 

2.3895 

9 

.0793 

.0982 

10 

.0072 

.7971 

11 

.0245 

.8316 

12 

.0251 

3.2304 

13 

.0469 

.4373 

14 

.0838 

2.5648 

15 

.0796 

.6377 
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From past record it is known that R - .95. Estimates of R PiIIi and corresponding confidence 
interval will depend on the model chosen and the method used lo compute the confidence 
interval. Thus, if X and Y are assumed independent, we have two independent samples of size 
15 each. Table 2 lists the values of R P2U) and confidence interval for the cases considered in 
section 5. We also use the notation of section 5 for convenience. 


TABLE 2 — Comparison of Various Confidence Intervals 
and Estimates of R 


Model Used 

R 

Confidence Interval 

G(l, 1) exact 

.9639 

(.9280, .9823) 

GO, 1) normal approximation 

.9639 

(.9300, .9978) 

G(2. 1) 

.9639 

(.9416, .9856) 

G(l, 2) 

.9952 

(.9896. 1.000*) 

G(2, 2) 

.9962 

(.9925, .9999) 

Binomial Procedure 

.9333 

(.8071, 1.000*) 

NP Procedure 

.9600 

(.7070, 1.000*) 


‘Computed .in being larger than 1 


Let us illustrate the calculation for a couple of cases. From the above data we have 
X ** .0509 and Y = 1.3602. If we assume p 2 — q 2 - 1 we have, using results in section 2, 

/?,, * -=r^ = .9639 
X + Y 

and corresponding 95% confidence interval for /? t ,, using normal approximation, is (.9300, 
.9978). 

On the other hand, if we used the exact distribution of R u . we can obtain the required 

y a 

confidence interval from an F-table since U = — — follows the F-distribution with (2 n, 2n) 

X “ 

degrees of freedom we obtain, after some simplification (.9280, .9823) as the required 95% 
confidence interval. 

From Table 2 it seems any of the first three procedures based on two independent 
exponential distributions or G(2. 1) (X gamma with shape parameter p — 2 and Y independent 
exponential) will be quite satisfactory. 
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OPTIMAL SELECTION FROM A RANDOM SEQUENCE 
WITH OBSERVATION ERRORS 

T. J. Siewari 

National Research Institute for Mathematical Sciences 
Pretoria, South A frica 

ABSTRACT 

A form of sequential decision problem is introduced in which options are 
presented in sequence, with no recall of rejected options (as in the secretary 
prohleml. but in which the value of each option may only be inferred from ex¬ 
periments. Decisions have thus to be made concerning both the acceptance 
and rejection of each option and the degree of experimentation. General pro¬ 
perties of the optimal policy are derived, and an algorithm is obtained for the 
solution in a special case. This special case suggests a heuristic rule for more 
general situations, the performance of which rule has been investigated by a 
Monte C arlo study. 


I. INTRODUCTION 

We consider a sequential decision problem in which a known finite number of options, 
having values drawn from a known probability distribution, are presented in random order to 
the decision maker who cannot observe the values directly. The decision maker has to perform 
a number of tests on each option (for instance by some form of sampling) in order to make 
inferences about its unknown value, and must finally decide on the acceptance or rejection of 
this current option before the next becomes available. We assume that the total number of 
tests that can be performed is strictly bounded above (owing, for instance, to budgetary con¬ 
straints). According to a criterion to be discussed in the next section, only one option may be 
chosen. 

The problem sketched above is a simplified representation of a number of practi: :l situa¬ 
tions. A mining development company may, for example, obtain options on various mineral 
deposits from time to time, but have to make a decision on each (either giving up the option to 
another or committing its resources to developing a mine at that site) before prospecting on the 
next site can even start. The value of each mineral deposit is, of course, unknown to the com¬ 
pany and must be estimated by bore-hole samples. These are, however, expensive, and a lim¬ 
ited number only can be afforded before a final decision has to be made. 

A similar situation may be faced in the procuring of major capital equipment, where 
expensive operational tests may be necessary to select between alternatives, and where, for 
practical or political reasons, negotiations with only one potential supplier at a time is possible. 
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These practical situations are generally more complex than the decision model discussed 
here. Nevertheless, a model of this nature can be used to gain valuable understanding of the 
real problem, as discussed in some detail in Stewart {91. 

It may be recognized that the problem described in the opening paragraph is essentially a 
generalization of the well-known ‘secretary’ or ‘dowry’ problem in the case of values drawn 
from a known distribution (for instance, the ‘full information game’ of Gilbert and Mosteller 
131), extended to allow for imperfect information (i.e., observation errors) on the value of each 
candidate. Note that we use ‘information’ in a different sense from that of Gilbert and Mos¬ 
teller, where ‘full information’ refers to complete knowledge of the distribution from which the 
values are drawn, but where observation of each value is implied to be error-free. The 
antithetical case of those writers is such that only relative ranking of candidates is possible, but 
in our model even ranking may be in error owing to observation errors (even though ‘full 
information’ concerning the underlying distribution is assumed to be available). 

MacQueen 141 has treated a somewhat similar problem, involving an unbounded number 
of options in a random sequence, but with a cost incurred in obtaining each new option. In his 
problem some imperfect information is available on receipt of each option, and further infor¬ 
mation can be obtained, at a cost, by executing a single test on the option. The present prob¬ 
lem differs from that of MacQueen in three main aspects: 

(1) the number of options is bounded; 

(2) the degree of testing is variable (under control of the decision maker), and 

(3) expenditure on testing is strictly limited. 

The p.'-'Mem' s also related to recent work on partially observable Markov processes (see, 
for instance, Smallwood and Sondik (71, Sondik (81 and White [111). In our case we have 
sufficient special structure to warrant special attention, but there are also some differences, 
especially in respect to the variable degree of testing and the budgetary constraints. 

In Section 2 the problem is formulated more precisely and the decision criterion dis¬ 
cussed. 

General properties of the solution under certain mild assumptions are developed in Sec¬ 
tion 3. In particular it is demonstrated that given a realization of a sufficient statistic y for the 
value of the option based on the tests conducted, there exist two critical values u and w such 
that the option is rejected for y ^ u, accepted for y > w, and further tested otherwise. These 
critical values depend on the numbers of options and permitted tests remaining, and on the 
number of tests already conducted on the option. 

In Section 4 a special case is introduced, for which a specific algorithm for computing 
these critical values can be obtained. Not only is this algorithm of use in its own right, but the 
special problem suggests an adaptation of its use as a heuristic for more general problems. 

Section 5 consists of a discussion of a Monte Carlo investigation of the behaviour of this 
heuristic. 
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2. PROBLEM FORMULATION AND DECISION CRITERIA 

We shall throughout the paper denote random variables by capital letters, and their 
specific realizations by the corresponding lower case letters. Conditional probabilities of the 
form PrlZl Y = y] will for convenience be written PrlZ|.y), and a similar convention will apply 
to expectations, probability distributions and density functions. In order to avoid technical 
difficulties we shall always assume that the probability distributions have densities. The random 
values Z„ and observations Y ni (defined below) will be assumed to be real-valued scalars. 

Now assume that a known number N of options are to be presented, and that the value of 

option n(n = 1. N) is the unobserved random variable Z„, drawn from a known 

probability distribution F(): 

F(z) = Pr[Z„ zl. 

Inferences concerning the actual realization of Z„ are to be made on the basis of observa¬ 
tions on r independent random variables ,, Y„ : . Y m . (The choice of r is discussed 

later). The probability distribution of Y„, conditional on Z„ = z„ is given by the known func¬ 
tion GCv„ ( |z„), with corresponding density ^(y^lz,,). Adopting a Bayesian approach, the prior 

distribution of Z„ is F(z„), while the posterior distribution is given by F„ r (z„\y„\ . y nr ) for 

r > I: 

(1) F„ r (z,,\y, A .v,„) = g(y n ,\z)dF„ r ^{z\y„\ . y, u . ,) 

where K is an appropriate integration constant and where for convenience F„ 0 (z) is defined as 
F(z). 


In setting up decision criteria two factors have to be considered, viz. the cost of observa¬ 
tions and the objective of selecting a satisfactory option. These factors could, in principle, be 
handled in a variety of ways, but for practical reasons it is necessary to adopt an approach that 
shows promise of yielding workable solutions. We shall thus assume the following structure for 
the problem: 

(1) a fixed total number of observations R will be allowed, which is equivalent to assum¬ 
ing a fixed budget; 

(2) the objective of the decision-maker is to maximize the expected value of the option 
chosen. 

Now let: 

y„ rf Sy „i. y,„) A expected optimal value of the option chosen, given that option n is under 

consideration Tthe first n - l having been rejected), that r observations Y „| = y „i. 

“ y,„ have been made on «, and that altogether p observations have been made on the 
first n options, 

and then let 

.i> A £ly H „.(.v„,..v,„)l.v„i.v,,., ,1. 
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In evaluating the expectation in the latter definition we use the posterior distribution of 
T,„, i.e., we use the probability density for Y m defined by: 

(2) O’Lv„i.>V-i) A J* Rl g(y\z)dF nr -\(z\y nX . y„.,-\)- 

For r > 1, p < R and n < /V, three options are available at this stage, viz. 

(i) to accept option /r, 

(ii) to make another observation on n; 

(iii) to reject option «, implying that option /; + 1 and the first observation on it become 
available. 

We thus have the following expression for y„,,,(y n \ .y„,): 

(3) TnrpOni.Jv) = Max [£{Z n 0„,. y„)\ 

^|.+ lO«l..V,,, )t. K, + I.l.,.+ |) 

where we note that A )1+ , does not depend on any observations. Then by invoking a 
dynamic programming argument (see, for example, Chow, Robbins and Sigmund 12], Section 
3 . 2 ), we have the following recursive relationship for r > 1, p < R and n < N\ 

(4) Kr P {y n i . y„.r- 1 ) “ J R , {Max \E(Z n \y„ .. y); 

iO’,,!. • • •. y„., i -y)\ K, + u. ( . + i)l*„.r- iO'U.i. ...y,,,-\)dy. 

For r = 1, p < R and n < N the same form as the above holds, with a null argument list for 
K,\,. and using #„,,(•) implied by (2) (from £„,,(•) = £(•)) in place of g „,_,(•). When n = N, 
this option must be accepted, and the expectation of its expected value after further sampling is 
the current expected value. When p = /?, either the current option or the (n + l)-th (without 
observation) has to be accepted. All these cases can be accounted for by the relationship (4) if 
the following definitions are used as boundary conditions: 

y.„ for r > 1 and all n < N 


V»\.k + \ = J" . zdFiz) A n for all n ^ N 


I,, * M 


for all p ^ R + 1 
for all p. 


The recursive relationship (4) now holds for 1 < n < N, 1 < p < R, 1 < r < p - «, 
and also defines the optimal policy which is to 

(i) select option n if 

£<Z,l.v„i.) > K,.i + i.,i+i(y,,i..v,„) and F„ +1 , ((+l ; 

(ii) take another observation on n if 

Vn,+\.„^yn\ .y»r) > £(Zjy„,.y„) and F„ + , l p+1 ; 
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(iii) reject option n otherwise. 

Quite clearly any general solution to (4) would be very difficult, if not impossible, to 
obtain in practice. Nevertheless, we obtain in the next section some general solution properties 
which may be useful in specific cases. An actually soluble problem is presented in Section 4, 
and the use of this result as a general rule is discussed. 

3. GENERAL 3GHJTION PROPERTIES 

Two assumptions regarding the distributions F and 0 will be invoked in this section at 
various times. These assumptions are stated first. 

ASSUMPTION (AI): The distributions are such that 

(i) given observations v„i.v,„, there exists a scalar-valued sufficient statistic y„, for 

(ii) y„, can be recursively generated (given a suitable definition of y n0 ) as a function of 
y„, r - 1 and y nr which is nondecreasing in each of its arguments; 

(iii) £lZ„|.v,„] is a nondecreasing function of v,„; 

(o. the posterior probability, Pr( Y m ^ .v\y tu \) is nondecreasing in v„, ( ...for all .v. 

Assumptions (i) and (ii) together imply that the dynamic programming recursion formu¬ 
lae (4) requires only .v,„ as stale variable, rather than the full observation vector. Assumptions 

(iii) and (iv) are introduced in similar spirit to assumption 2" of MacQueen [4], The require¬ 
ment is that the probability mass of the observations should follow the true underlying value. 
More specifically (iii) requires that posterior inferences on z„ should in turn follow the observa¬ 
tions, while conversely (iv) requires that posterior probability mass on future observations 
should follow the inferred underlying value through the sufficient statistic y„, r -\- Assumption 

(iv) is, in a sense, analogous to the ‘increasing failure rate’ assumption of reliability theory. In 
our case, however, it simply implies that high observation values should not decrease the poste¬ 
rior probability of future high observation values (i.e., a form of consistency assumption). 

In fact, (Al) may be expected to hold quite widely. It holds, for example, when 
Z„ ~ N{ 0, 1) and Y,„ — N(;„, <r : ), in which case; 

1 v* 

'*■ 7 , 5 '” 


or 


and 


y,„ - {(r - 1 )y„, -1 + y„,}/r 
y „o - 0, say (arbitrarily) 


EZ. 


i_ r - „ .] (*■ “ IbVr-t o- 2 («r 2 +r) 

- ^T7| v ■ ~ 1 • »■ + ,-! ' 


* >m 
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The normal case also satisfies the second assumption below, which is really an extension of (iii) 
and (iv) of (Al), now requiring a certain consistency between the probability mass of the 
observations and the true value in the tails of the distributions as well. 

ASSUMPTION (A2): (Al) holds, and for any e > 0 there exist >>| and y 2 such that 
( y m < Ti) and { Y,„ > y : | have nonzero probability and for any m > n 

(i) y„, < >'| implies 

Pr(Z m > z„|yj =/ r , Fjz\y nr )dF(z) > l - € 

(ii) y,„ > y 2 implies 

Pr[Z,„ > z„\yj < e. 

We now consider certain general properties of the solution. First, however, we prove the fol¬ 
lowing lemma. 

LEMMA: If (Al) holds, then P„, ( ,(y„.,.|) (= V mv (y„i.y„. r -|), see comment after 

assumption (AD) is a nondecreasing function of y„, i, which is, furthermore, such that: 

D,„„ly,,r ,) A l£Z„|y„, ,] - V,„ r (y lu ,) 

is a nonpositive, nondecreasing function of y„, (Note that by virtue of (Al) (i), conditional 
expectations depend only on y„, |.) 

PROOF: By assumptions (Al) (ii) and (Al) (iii) it follows that for given y nr » y: 
£[Z„|y,,, yj = £(Z„|y„,(y„., y)] 

is a nondecreasing function of y„ , i and y. 

The boundary conditions on K„,, imply that for any r(1 ^ r < R) and given y,„ = y. 

>\,k(.i'm.>’\<) = y\,K^y\., i. y) 

- £(Zjy v , y\ 

while for n < A 

<(y„, |, y) = Max(£iZ„ly„,<y„, j. >)}; m1- 

(Uere and elsewhere in the proof y,„ K (y„, t , y) is a compressed form of y„,/f (y,„ (y„, r i. y)) 
and similarly for the other functions.) 

Thus, for all n < N and r < R and a given value for y, y nr R (y„|.y) is non-decreasing 
in 1 and y. Now define >', y„,_i and v. Now define 

(y„, i, y) A £{Z„|y„, |. y} - y,„K (y„., i. y) 

= MinlO; £{Z„|y,„(y, u y)} - /a). 

C learly, 8 y) is also nondecreasing in y„, , and y, and is nonpositive. 

Now define the following probability density function on y: 

, (y) A y„, |(y|y„ , ,) 
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with corresponding distribution function G„, |(y). This distribution is defined to be indepen¬ 
dent of any perturbation of y„, i around the nominal value. 

Since >(•) and 8() are nondecreasing in y„, i for every y , it follows that their expecta¬ 
tions with respect to the distribution defined by g, u ( (-) are also nondecreasing in y„ r Thus, 
defining for any y' . (not necessarily the nominal value y„, |): 

A J„ y.nKty ,.r |. T > X,,, |(.V)^ and 

,) A f 8„, R (P’ v)x„, i (v)dv 

= ■ , 

we have for any h > 0: 

V mK (y„, |+/i) > K„k(.v„,, |) = V,„k (y„, |) and 

i +*) > D„ rft (y„, i> =/ K ,§,„/<(T,,, i• 3 ")Xui |(yly„., ()</>> 

= fiZjy,,, ,) - F,„«(y„, ,) 

= D„, K (y„, ,). 

Now consider the expectations of >„,«(•, •) and 8„ rR F. •) when y has distribution function 
f'nr-iO'l.T’n.r -1 +*)• By assumption (Al) (iv), if this distribution differs from G nr _|(y), the 
difference is a shift in probability weight to higher values of y , and thus, since -y nrR (-. ) and 
8 nrR (•, •) are nondecreasing in y. we have for h > 0: 

f V n,R for- I +h) > V„rR(y„.r- 1 + * ) ^ (y„. r _ ,) 

V D„rR O',.,-1 +h) > b„ rR <d n .,-\ +h) > D„ R (y„,.,). 

Furtherrrtore, taking expectations, it is also clear that D,„« (y„, |) inherits the nonpositivity of 
8„ (R (y„. r ,'i 1 y). The lemma thus holds for p = R. For other values of p, the F„, (J (-) values can 
be obtained by backward induction from the F,„ R (-) computed above and from the boundary 
condition. 


Fm () = p, p ^ R + \ . 

We thus assume that the lemma holds for n,r + I, p + l, i.e.. that f 7 ,,, + I|l+1 (y,„) and 
D„ , (y w ) are nondecreasing functions of y,„ and that: 

< 0. 

Subtracting the expression for y,„,,(y„, i, y) given by Equation (3) (where, as above, 
y„i.y„, i is replaced by y„., i and y,„ = y) from E (Z„|y,„(y lu y)|, we have 

8,„,,(y,,, y) = Min {0; E |Z„|y, ir (y„, >•)} - K,, tl ( , + ,{y,„(y ll( _,, y)); 

£(Z„ly,„ (y„, y)} - F„>i i, |1+ |l. 

= Min {0; D „, + ,{y,„ (y„, y)}; 

f|Zjy w Cv,_,. y)) — F fl+ |.,. + ,}. 

Since F„ + , , ()+) is a constant not dependent on y„, ) or y we have from (3) and the above, and 
using (Al) (ii) and (Al) (iii), that y,„ (l <y„. r i, y) and 8,„ ( ,(y„, y) are nondecreasing func¬ 
tions of y„ , i and y, and that 8„, ( ,(y„, y) is nonpositive. In the same way as above it can be 
shown that F,„,,(y„., |) and D„ ri ,{y nr |) are nondecreasing in y„, ) and that 


VOL 28. NO 3. SEPTEMBER 1981 


NAVAL RESEARCH LOGISTICS QUARTERLY 







400 


T J STI.WART 


0,, p Ov,-i) < o. ii 

The lemma follows by induction. 

Lemma 1 may be seen to be a special case of a wider class of results derived by Topkis 
[101, where conditions under which optimization problems yield solutions which are isotonic in 
a parameter are discussed. In the present case, however, not only is the direct proof relatively 
simple, but it directly yields the structure of the optimal policy. In fact, by the optimality prin¬ 
ciple of the last section, there is implicit in the proof also the demonstration of the following 
theorem. 

THEOREM 1: If (Al) holds, then the optimal policy is defined by numbers and w,,,,, 
for all r , p such that 


(they may be equal), and 

(i) option n is rejected if y,„ < 

(ii) option n is accepted if y,„ > 

(iii) a further sample is taken otherwise. 

This is a result which is intuitively fairly obvious although difficult to prove, and it is simi¬ 
lar in spirit to related results of Rosenfield 16] and MacQueen [4], General conditions for the 
optimality of structured classes of policies, such as that of Theorem f, have been derived by 
Porteus [51. Thus, as for Lemma i. Theorem i, is a special case of a more general result. Our 
derivation has, however, allowed the specific form of structuring to be developed construc¬ 
tively, and thus Theorem 1 is in a form suitable for direct practical implementation. 

This structure of the optimal policy is important for a number of reasons. Certainly it is 
easier to store i/„,„ and w„ rt , values than to store V,„,,(■) for all values of its argument. In some 
cases it may even be possible to compute u hrp and w nrp directly, although we have not found 
such an example. Furthermore, once this structure of the optimal policy is known, various 
heuristics.can be proposed for the problem, based on assumed or heurislically derived values 
for the u mp and tv,,,,,. 

We nov. state and prove a further property of the optimal policy under the stronger 
assumption (A2). This result is given here not only because it is an aid to intuitive under¬ 
standing of the problem, but also because it may suggest heuristic derivations for the u mp and 
w mp above, based on approximations for K„,,(.v„., i). 

THEOREM 2: If (A2) holds and y,„ € l y,, y„l for all n and r (we may have y, — —«*» 
and/or y„ = °°), then 

lim y w ,,(y„. r i) = E„ +) | )1+ , 

lim V Xv„, |) = E l£|Zjy w (y„, i. .v„,))l.v„, il. 

(Note that, although the second expression above is in a rather cumbersome form, it is in most 
practical applications quite easily evaluated). 
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PROOF: By assumption, for small enough y„, t it will be optimal to reject option n 
immediately, giving: 

v™,.Gvr-i. y ) = ^i+u.p+i 
for all y„ , i and y, and the first limit holds. 

Similarly, for large enough y„, t , option n is immediately accepted, giving 

y„„.ly,,r \.y) = E[Z„\y nr . u y), 

and the second limit follows. II 

In this paper we have not attempted to develop any heuristics on the basis suggested 
above. Rather, we introduce in the next section a particular case in which the exact solution 
can be computed. Not only is this case of some interest in its own right, but it suggests 
another heuristic approach for the general case. The results of a Monte Carlo study of the per¬ 
formance of this heuristic are presented in Section 5. 

4. BETA-DISTRIBUTED VALUES WITH BINOMIAL TRIALS 

In this section we consider the case in which the Z„ are independently distributed accord¬ 
ing to a Beta distribution, with parameters a and j3, i.e., Z„ € [0, 11 and F( ) has density: 

rt-) „ * A a t.P -L „<i _ ,\ii i 

' l'(a)T(/3) ’ 

The observations Y,„ are of a simple binomial success/failure form, with success probabil¬ 
ity z„. This is a somewhat simplified model of many real situations in which z„ represents, for 
each option, a certain degree of satisfaction, and the y„ k represent either a simple sampling 
(e g., defectives in a batch) or the results of expert opinions (each declaring the option ‘satis¬ 
factory’ where z„ is now the fraction of alt experts who would find the option satisfactory). 

Since y,„ € (0. 1}, we can define 


« + £ y„, 

i -1 


y„.r I + TV- 


Then, a posteriori, the uncertainty in z„ can also be represented by a Beta distribution with 
parameters y,„ and (a +13 + r - y,„), i.e., 

E Z„|y„, - yj(a + /3 + r) - Pr( K„ f+ , - 11 y,„). 

Using the same boundary conditions as in Section 2, we thus have for n < A, r < p, p < R. 

Ct +0 + r- l-jv-i [ j y„.r-l \ ir I 

+ a +/3 + r— 1 | MaX L +/3 + / .’ K »r+I.P+tO',.r-t). «+l.l.p+l| 


a ^.^j rr ' ^».r+l.p+tO'ii.r-t)'* Ki+l.l.p + 1 
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Now because the state space of y nr is finite, i.e., y lir € (0, 1.r), it is in principle possible to 

compute V M(I (y„, ( ) for all values of w, r, p, and J„, |. This would be achieved by starting for 
each n < N with each value of s in 1 < s < R — n, and then computing by a backward recur¬ 
sion for j = 0. I. (s - I), k„, ( , (y„, ,) where r = s - j and p-R-j for all 

.v„., i € (0, 1. r — 1). Note that in doing so we may discard all K,., + i.,,+i( - ) values after 

computing the F„, () (-) values and storing the u„, ( , and h„, ( 1 values. Apart from this the only 
other intermediate results requiring storage are the (R - n ) values of f / H+1 ((l+) (p = n + 2, 

n + 3. R 4- 1). Thus, at most, 3 (R - 1) intermediate storage positions are necessary. 

Finally, the policy variables and must be stored, but this need not be in-core; approxi¬ 
mately 2NR : values would be generated. 


In order to achieve this compulation, the following number of evaluations of (5) are 
required: 

N I K s „ \ I * " etc 4- n 

I II (s~J)= Z I 


1 s 1 

J I 


(R-n) (R - n + 1) (2R -2/t + l) (R- n)(R~n + ]) 


-jy I l2R(R + \)(R+2) - 2<3/? : + 6/?+2)/;+6(/f + !)«-’-2/f‘J. 



I 


= R(R + I) (R +2) (A- 1) _ (}R : + bR +2) (A - \)N 
12 12 

(R + \) (A- \)N(2N-\) _ A 2 (A- l) 2 
12 24 

For moderate values of N and R (e.g., 10 and 10\ respectively,) this computation is indeed 
quite feasible, since evaluation of (5) is extremely simple (three additions/subtractions, six 
multiplications/divisions and four comparisons). 

Although the solution is not very elegant, its feasibility is of interest. In particular, we 
consider the case in which a - )3 = 1, i.e., the case of uniform distribution on [0, 1]. For a 
known distribution function G( ), Z„ can always be transformed into a variable Z,\ which is uni¬ 
formly distributed. Maximizing the expected value of the Z,', chosen is equivalent to maximiz¬ 
ing the fractile value of Z„' and thus of Z„ (which also minimizes the probability that one given 
later option will improve upon option n). This is in fact a quite satisfying objective in the gen¬ 
eral case (cf. the ‘uniform game,’ paragraph 5a of Gilbert and Mosteller (31). 

Now if the Y' m represent, as above, binomial trials on the uniformly distributed Z' n , then 
any event of the form { Y^ r ^ f) is equivalent to the event | [estimated probaHity that 
Z,„ < r„, for any given m > «] < £/(r + 2)}. 

For general observations y,„ on z„ this probability can also be computed: it is in fact given 
by rj„, where 

(6) v„, - f , ll - F„(r|v„,))</F<r). 

(It is easily confirmed that in the uniform/binomial case rj,'„ - y,„/(r + 2).) This suggests that a 
heuristic policy may be defined by the following: where and are the critical values 
computed in the binomial case and with a - 0 - 1, 
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(i) option n is rejected if rjnr ^ u' lt ,J(r + 2); 

(ii) option n is accepted if yj nr > (r + 2); 

(iii) a further sample is taken otherwise. 

5. MONTE CARLO RESULTS 


In order to test the heuristic suggested at the end of the last section, a series of Monte 
Carlo studies were performed. The example selected for these studies was chosen more to 
demonstrate performance under fairly extreme conditions than to represent any particular prac¬ 
tical situation. We give first a description of this example. 


The values Z„ are drawn from an inverse gamma distribution with shape parameter 3 and 
scale parameter 1, i.e., if we define X„ *= l/Z„, then the distribution of X„ has density 

<£(x) = ~ xV\ 

The observations Y,„ are drawn from the exponential distribution with mean Z„, i.e., 
sCvIz,)- lexp(-j'/z„)]/z B . 

A sufficient statistic for z„ is thus 


y,» = 


£>«/ 

/-i 


It is not difficult to show that (Al) (in fact (A2)) holds. The value of ij,,, of the previous sec¬ 
tion is easily obtained, although in a form slightly different from that of Equation (6). We note 
first that the posterior distribution of x„ ( note, not z„) is also of the gamma form with density 
defined as 4>,„ix) given by. 


(x) = 


0v+l)' + ' x 2+r expl-Ov+Dx] 
n 3 + r) 


The probability that Z,„ < z„ for any given m > n, is equal to the corresponding probability 
that X,„ > x„. Conditional on x„ ( i.e., on z„) this probability is given by 


PrU„, > X n \X„ = x„] = 


■x'e 'dx 


1 + x„ + — x,f| exp(-x„). 


Taking the expectation of this conditional probability with respect to the posterior distri¬ 
bution for X„ conditional on y,„, we obtain the required expression for rj w as follows: 

Ov+D w ' expl-(5',„+2)x] 


/; 


1(3 + r) 


x + jr 


' + 1 x 4 
2 


dx 


y ,„+1 


y„,+2 


1 + 3 + r + (3 + r) (4 4- r) 


+2 


20v +2) 2 


Vv ;th these results the proposed heuristic can be applied. 
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In ihe studies performed, R - ION throughout. Now it is important to note that simply 
to take 10 observations on each value, and to use the mean of these as the true value in one of 
standard models (as, for instance, in Gilbert and Mosteller [3]) can lead to substantial errors. 
For example, around the mean value of X„ (i.e., 3) the variance of the estimate of z„ is 
0.03333, and the one standard deviation limits of this estimate is (0.1507; 0.51591. In fact, this 
range within which the estimate is most likely to fall, covers approximately 65% of the probabil¬ 
ity mass of Z„. Thus, even if all N options could be seen simultaneously, in ordering the 
options quite considerable errors could occur. 

The Monte Carlo studies with the heuristic rule were carried out for four values of N, viz. 
4, 6, 8 and 10. In each case 1,000 sets of A values were generated by a standard pseudo¬ 
random number generator. The cutoff points and had been computed previously and 
stored. For each set the rule was applied, and the rank and value of z„ for the option chosen 
was recorded. In this way the following comparisons could be made with optimal performance 
in perfect information models: 

(a) Mean fractile of the option selected (i.e., the expected probability that the chosen 
option exceeds a random draw of Z m ; if Z„ is transformed to a random variable, U„ 
say, which is uniformly distributed on (0. 1] then this corresponds to EU„ for the 
option selected). The observed mean can be compared with the corresponding 
optimal expected value without observation error ((31, Table 11). It may be recalled 
that maximization of this value is the objective invoked as a basis for the heuristic. 

(b) Frequency of selection of the best option. This can be compared with corresponding 
optimal probabilities in the standard cases with no observation error when the under¬ 
lying distribution is either known <{3J, Equation (3c-1) and Tables 7 and 8) or unk¬ 
nown (and ranking only is possible; (3) Table 2). 

(c) Mean rank of the option chosen (where 1 = best). This can be compared with the 
corresponding optimal expected rank, if ranking only is allowed (111. Equations (6) 
and (7)). 

These comparisons are shown in Table 1 

Clearly the heuristic rule compares well with optimal rules when there are no observation 
errors. In particular, the rule performs substantially better than the optimal rules based on 
error-free ranking of the options only. Of more importance, however, is the fact that perfor¬ 
mance compares favourably with optimal rules based on full information on the underlying dis¬ 
tribution and perfect information on the values in hand. This advantage is particularly evident 
for the mean fractile of the option chosen, the basic objective of the heuristic. 

6. CONCLUDING COMMENTS 

We have introduced a class of sequential decision problems which generalize the "secre¬ 
tary" or "dowry" problems. Apart from general solution properties, an exact solution has been 
found in principle for a particular case. What is more important, it has been shown that the 
solution in this case can form the basis of heuristic rule for more general situations, which rule 
has been validated by means of a Monte Carlo study. Admittedly, the rule is computationally 
convenient only for rather small numbers of options, but it must be noted that the motivating 
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T ABLE 1 — Comparison of Performance of the Heuristic Rule and Optimal 
Performance with Error-Free Information 



Number of options (N) 

Mean fractile of the option 

4 

6 

8 

10 

0,727 

0,786 

0,818 

0,853 

selected. 

estimate, case H 

(standard deviation of estimate) 

(0,007) 

(0,006) 

(0,006) 

(0,005) 

optimal expected value, 
case 0, 

0,742 

0,800 

0,836 

0,861 

Probability of selecting 
the best option: 





observed frequency, case II 

0,607 

0,580 

0,516 

0,526 

(estimated standard deviation of 
frequency) 

(0,015) 

(0,016) 

(0,016) 

(0,016) 

optimal probability, case O. 

0,655 

0,629 

0,616 

0,609 

optimal probability, case 0, 

0,458 

0,428 

0,410 

0,399 

Mean rank of option 
selected 





estimate, case H 

1,587 

1,734 

1,921 

1,989 

(standard deviatioi of estimate) 

(0,027) 

(0,035) 

(0,043) 

(0,049) 

optimal expected rank, 
case O-, 

1,875 

2,217 

2,400 

2,558 


( .inc II heuristic rule of Section 4. estimates baser! on the Monte Carlo study 
C ase O, optimal rule given perfect information on \alues drawn from a known distribution 
( ase <) optimal rule given perfect information only on ranks of the observed options 


examples in the introduction (in which costly examination of each option is required) will gen¬ 
erally involve rather few options. In these cases the model and solution would appear to have 
some applicability 
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ABSTRACT 

In this paper we analyze optimal search strategies in an environment in 
which multiple, independent targets arrive and depart at random. The analysis 
revolves around a continuous time differential equation model which captures 
the time dependent nature of the search process. We explore the impact on 
optimal strategies of nonzero travel limes between regions as well as differing 
target arrival rales. We derive simple closed form expressions for determining 
if only one region should be searched. 


INTRODUCTION 

In this paper we present a differential equation model used to explore optimal search stra¬ 
tegies in an environment which involves: 

1. multiple independent targets 

2. random arrival of targets 

3. random departure of targets 

4. different target types (mean duration) 

5. time lost in travel between noncontiguous regions. 

A wide range of target types exhibit characteristics similar to some or all of the above. This 
research evolved initially from research into crimes but has application to the following target 
types, a) smugglers, b) terrorists, c) transiting submarines, d) infiltrating enemy forces and e) 
groups of attacking aircraft. 

After a brief literature review (see 113], [15], and 117] for more extensive reviews), we 
introduce the basic differential equation model and apply it to a series of increasingly more 
complex problems. The analysis considers only cyclical search strategies and focuses on the two 
region problem. Our goals are: (1) to determine the optimal cyclical strategy, and (2) to 
develop simple analytic expressions for when a region should be excluded from search. 
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The problems we address are as follows: 

(1) Two regions with equal target arrival rates, no time lost in travel between regions. 

We determine for this problem the optimal cycle length and the magnitude of the impact of 
using optimal search strategies. 

(2) Two regions with equal target arrival rates, time lost in travel between regions. 

For this problem we again determine the optimal cycle length and also develop an analytic 
expression for when to search only one region. This expression is a function of the travel time, 
and the departure and detection rates of the targets, but not the arrival rate. 

(3) Two regions with unequal target arrival rates, no time lost in travel between regions. 

We determine optimal cyclical policies and an analytic expression for when to search only the 
high target region which is a function of only the ratio of the departure to the detection rate. 
In addition, we explore the impact of constraining the minimum time to search a region. 

In all of the above examples we consider only a single target type and only two regions. 
We close with a brief discussion of: 

(4) Two regions with multiple target types. 

(5) More than two regions. 

Throughout the paper we evaluate the limiting properties of many of the equations and, 
for example, study the impact of infinitesimally short cycle times. Although these cycle lengths 
are not achievable in the real world, the results provide important bounds on the difference 
between optimal and nonoptimal strategies. 

1. LITERATURE REVIEW 

The pioneering work by Koopman [9], {10], and [11], along with the work in the discrete 
analog by Charnes and Cooper [3], has laid the foundation for the continuing expansion of 
search theory. The later work which is most directly applicable to our area are those papers 
which explore the sequencing of search effort. Blackman and Proschan [2] analyzed the optimal 
sequencing of search effort among a series of regions into which targets arrive randomly. Gil¬ 
bert [6], and later, Kisi [8] analyzed a two-box search problem with the objective to minimize 
the expected time until detection. However, their work is not extendable to situations with 
departing targets [4], Moore [14] analyzed the search for one randomly arriving and departing 
target in n regions solely from the perspective of how much search effort to allocate to each 
region, but not the sequencing problem. The paper closest in orientation to this work is that of 
Barnett [1]. In his problem, targets arrive in a Poisson process and survive for a random dura¬ 
tion. Barnett, in considering a discretized search, (1) proves optimal strategies are cyclical, a 
fact we use in applying the differential equation model, (2) finds, analytically, the optimal 
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sequence of search, and (3) generates a sufficient condition for excluding regions from the 
optimal search strategy. We develop analogous results while introducing a more flexible con¬ 
tinuous search problem which allows for travel time between regions and the possibility of 
searching a region and not discovering a target that is actually present. 

2. DIFFERENTIAL EQUATION MODEL 

The model consists of two differential equations for each region, one to describe the sys¬ 
tem when no search is in progress in that region, the other to describe the system when a 
search is in progress. 

Let S - the state of the system (expected value of the number of targets present). 

A — the rate at which targets arrive (assumed independent of the state of the system). 

Targets depart from the system in either one of two ways. Either targets leave because they 
have finished their task or because they have been intercepted during a period of search. 


F = the constant of proportionality for the rate at which targets finish their tasks. (This 
will depend upon the duration of their task.) 

/ =■ the constant of proportionality for the target interception rate. (This will depend on 
the size of the region and the observability of the target.) 

Using the above parameters (S, A, F, /), we can write the following equations to 
describe the changing system. During a period of no search, the system is described by 

(1) - 5' - A - FS. 
dt 

The equation states that the system is changing because targets are arriving at a rate A and are 
departing, as a result of finishing, at a rate FS , proportional to the state of the system. The 
solution of the differential equation is 

(2) C, exp(-Ff) + A/F. 

A IF represents the steady state number of targets if no search were carried out. 

During a period of search, the equation becomes 

(3) —■• A — (F A- l)S 
dt 

whose solution is 

(4) S - C 2 exp(-(F A-l)t) A-A/(F A I). 

A/{F + /) is the steady state number of targets during an unending period of search. 

These equations assume that the arrival process of targets is Poisson and that the duration 
of a target's presence is exponentially distributed. 5 is therefore the expected value of the 
number of targets. This can be shown by setting up queueing type equations for the probabili¬ 
ties P„ , where n is the number of targets present at any one time. 
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3. TWO REGIONS—EQUAL TARGET ARRIVAL RATES-NO 
TIME LOST IN TRANSFER 

The first problem to which we apply the differential equation involves two identical 
regions. For now we will focus on the optimal cycle length and consider only equal allocations 
of search effort to the two regions; later, we will show that in fact the global optimal strategy is 
symmetric. In addition, we will consider only simple cyclical strategies of the form X minutes 
of search in R\, followed by X minutes in R2 and then back to R1 for X minutes, and so on. 
This second assumption is motivated by a number of earlier search theory papers ll, 6, 8] in 
which the optimal search strategies were cyclical. 

Our objective will be to minimize the total average expected number of targets in the 
combined regions. However, because the regions generate independent target arrivals and 
departures, they will be treated separately as experiencing alternating periods of search and no 
search. In addressing this problem we will, in addition, answer simultaneously the following 
question: 

Given that a searcher is unavailable 50% of the time, what is the distinction between short 
numerous intervals of search and a few long intervals of search? 

The three steps common to this and the later examples are outlined as follows: 

1. Two differential equations are defined for each region. One to describe the dynamics 
of the region while it is being searched, the other to describe the region while no 
search is going on. 

2. For each region, two boundary conditions of the following form are established. The 
target level in each region at the end of a period of search must equal the level at the 
beginning of a period of no search and vice versa. These conditions are simply a 
continuity constraint on S, the number of targets present, not allowing discrete shifts 
in the values of S as a result of the searcher entering or departing a region. Once the 
boundary conditions are defined, each of the constants in the differential equations is 
determined. 

3. The average target level, S, in the total area is calculated by integrating each of the 
four equations over its respective period of search or no search, summing the four 
values and dividing by the cycle length. The resultant expression for S is a function 
of the parameters of interest in each of the particular examples and is subsequently 
analyzed to determine an optimal strategy. 

In both regions, R 1 and R 2, the two equations describing the periods of search and no 
search are just the previously defined Equations (1) and (3) and whose solution are Equations 
(2) and (4). 

The continuity constrainton S is used to generate the following equation: 

(5) C 2 exp (—(F + l)X) + A/(F + /) - C, + A/F. 
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The left-hand side represents the end of a search period of duration X minutes and the right- 
hand side the beginning of a no-search period. A second analogous equation can be set up for 
the end of a period of no search and the beginning of a period of search. These equations are 
then solved to find C| and C 2 as functions of X. The expected target level in each of the 
regions at any instant in time can now be written as an integral function of only one parameter, 
X , the half-cycle length, which yields 

(6) ST = - (CJOFX)) x (1 - exp(-FX)) + A/2F 

~ (C 2 /2(F + l)X)) x (1 - exp(-(F + l)X) + A/2(F + /). 

The average for the entire region, S, is merely twice that of any individual region. 

With some algebraic manipulation of the derivative, we have proved [4) that S increases 
monotonically as X increases. In other words, the average expected target level in each region 
decreases as the frequency of transfers between the two regions increases. This result is 
directly comparable to that of Gilbert (6J. This result also implies that if search were scheduled 
in long blocks of time (i.e., increase X ), and no penalties were incurred for switching from 
search to no search the model predicts a decrease in the probability of interception. 

Magnitude of Impact of Short Cycles 

To determine the magnitude of improvement produced by the shorter cycles, we deter¬ 
mined the average number of targets present as the cycle length, X, approaches infinity. 

(7) ST = 1/2 (A/F) + \I2(AI(F + l)) = A (2 F + /)/2F(F + I) 

For short cycles as * approaches zero. Equation (6) becomes 2A/OF + /) and thus the reduc¬ 
tion of the average target level for short cycles as compared to long cycles is 

(8) \I(2FI l 4- I) 2 . 

This expression, not surprisingly, does not depend on the arrival rate. A, and depends only on 
the ratio of the departure rate, F, to the interception rate, /. Thus, for example, if the depar¬ 
ture and detection rates were equal, infinitesimally short cycles would reduce the target level by 
1/9 as compared to long cycles. Table 1 summarizes the reduction for a range of ratios. As the 
departure rate grows (shorter mean duration) relative to the interception rate, the impact of 
shorter cycles decreases rapidly. 

If, as an alternative, we measure the relative impact of long and short cycles against a no 
search policy then the reduction due to infinitely long searches is //2(F + /) and for short 
cycles it is llOF 4- /). If, for example, / = F then short cycles reduce the target level by 1/3 
more than long cycles. The probability of interception under each policy is directly related to 
this proportional reduction in target level. For infinitely long cycles the fraction of targets inter¬ 
cepted while searching is just // (F 4- /) but we search only half of the time, (//2(F + /). 

For infinitesimally short cycles during any target’s lifetime, the searcher and target will be 
in the same region half of the time. Consequently, the probability of interception is 

(9) f Fexp(-Fz) x (1 - exp (—/r/2)) di. 

•'ll 

This yields [I/OF 4- /)], which represents the reduction in target level for infinitesimally short 
cycles. 
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TABLE I — A Comparison of Short and Long Cycles With 
Regard to Interception Probabilities and the 
A verage Expected Number of Targets Present 


R = Fil 

Reduction in 
Target Level 

Due to Short Cycles 

Probability of Interception 

1 

I - SJS, 

.111 

Short Cycles 
.333 

Long Cycles 
.250 

2 

.040 

.200 

.167 

3 

.020 

.143 

.125 

4 

0123 

.111 

.100 

5 

.0083 

.091 

.083 

10 

.0023 

.0476 

.0455 

20 

.00059 

.0244 

.0238 

30 

.00027 

.0164 

.0161 

40 

.00015 

.0123 

.0122 

50 

.000098 

.0099 

.0098 


4. TWO REGIONS - EQUAL - TARGET RATES - TIME LOST IN TRANSFER 

In this section we modify the previous example by introducing a parameter L, which 
represents the time lost from search while traveling between the two regions. We will use the 
differential equation model to address two questions. 

1. What is the optimal value of X , the time spent in R 1 before switching to R2? The 
optimal value is obviously no longer infinitesimal. 

2. Is there a simple analytic expression which specifies for which values of L it does not 
even pay to switch regions? 

In tackling the problem with the differential equation model, we will again assume a cycli¬ 
cal strategy (both Kisi [81 and Gilbert [6! have cyclic strategies) and that the search effort will 
be divided equally between the two regions. 


The key to our analysis is that optimal solutions which limit the search to only one region 
will appear in the analysis as solutions in which the optimal value of A" is infinite. 


In this second problem, which ascribes a penalty for switching regions, the basic equations 
which describe the periods of search and no search do not vary from those of the previous 
example (Equations 2 and 4). The introduction of the switching time, L, does affect the con¬ 
tinuity boundary conditions and changes Ci and C 2 since a no search period in each region has 
a duration of ‘X -I- 2 L' minutes. The average SI is calculated as in Equation (6) except that 
the no search time period extends over a time period of X + 2L minutes and that the cycle 
length is now 2X + 2 L. The resultant expression is 


( 10 ) 


SI - A/F - [AIX/2(X + L)F(F + /)! + \A/2(X 4- L)] 


x ( I/FIF+ I)) 7 x 


1 + 


2 - exp(-(/ + F)X) - exp (-(AT -I- 2L)F) 


exp (-(2 F + l)X - 2FL) - 1 
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Once values have been assigned to F, I and L , this equation is a function of only one vari¬ 
able, X, whose optimum is easily determined. In Figure 1 the optimum value of X is shown for 
a range of L, F and R , where R is the ratio of F, the departure rate, to /, the interception rate. 
There are graphs for two values of F, one for F equal to 20—targets last an average of three 
minutes—and one for F equal to 10, with R as high as 20, and as low as .01, which would mean 
that targets are intercepted at a rate one hundred times as fast as they depart on their own. The 
lower values for R were included less for realism than to display how the curves behave as R 
approaches the limit of zero. 

R= 20 10 5 2 1 5 1 01 

X 4f | | | / / I it 


F=20 per hr 



6 1 2 18 2 4 3 0 3 6 minutes L 



4 2 minutes L 


Fioure 1. Optimal value of X as a function of L 


The most striking characteristic of these curves is their asymptotic nature and the location 
of the asymptote. 

The asymptote is always less than the mean duration of the target, 1/F, and approaches 
the mean duration as R approaches 0. In addition, for a given value of L as the ratio, R, 
decreases, the optimum X also decreases; as F decreases (from 20 to 10) the value of the 
asymptote for a given R increases. 

All of the above points will be addressed more formally in the succeeding section in which an 
analytic expression for the asymptote as a function of Fand R is derived. 

When to Search Only One Region 

The development of an analytic expression for the asymptote as a function of Fand R 
builds on the observation that SI, the average target level will either be (1) a monotonically 
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decreasing function of X, or (2) a unimodal function. We have no formal proof of this but 
extensive computer analysis of SI for a wide range of values of F, R and X supports this claim. 
In both cases the limiting value as X tends to infinity is the same. Because of the complexity of 
the derivative, we found that the simplest way to determine monotonicity was to explore the 
limiting behavior of the derivative. If the derivative approached zero from the negative direc¬ 
tion as X increased to infinity then the function was monotonic and the optimal strategy was to 
search only one region. It can be shown that after excluding all components of the derivative 
which approach zero faster than \/(X + L) 2 and consider only the direction of the derivative, 
we find the expression of interest to be 


( 11 ) 


lim - L - 


F(F + /) 


1 - [2 - exp(-(/ + F)X) - exp i-F{2L + A'))] 


with D = exp(-2 F(X + L) - IX) - 1. 


Its limit is 

(12) - L- I/FU + F) + 2 l/FU + F) = - L + [1 /F] x [l/UF/l) + 1)]. 
This will be negative and thus the optimal value of X infinite whenever 

(13) L [l/F] x [1/((F//) + 1)1. 


This confirms our earlier graphical analysis. The second component [l/HF/D + 1)1 is 
always less than 1 and approaches 1 when R , the ratio of F to /, approaches 0. Thus, the 
asymptote for each curve in Figure 1 is less than the mean target duration, 1 /F, and approaches 
the mean target duration as R approaches 0. Of perhaps greater significance is that even if L is 
orders of magnitude shorter than the mean duration, the optimal strategy might be to search 
only one region. For example, assume targets depart at nine times the rate they are detected. 
If the travel time is more than one-tenth the average duration of a target (e.g., 12 seconds for 
two minute targets, 18 seconds for three minute ones), the optimal strategy is to search only 
one region. 

5. DIFFERING TARGET ARRIVAL RATES 

The final example involves two regions, RH and RL. In RL targets arrive at a rate A and 
in RH , the high target region, targets arrive at a rate M * A, with M greater than 1. In both 
regions all targets have the same departure and interception rates, F and /, respectively. Once 
again, our analysis will revolve about cyclic policies, this time of the form, X minutes in RL 
followed by A x A minutes in RH. 

Although one problem of interest is the finding of the (A, X) pair which minimizes the 
average expected number of targets in progress, the discussion will not be limited to that, since 
in all instances the optimum is approached as X tends towards zero, an unimplementable 
optimum. Therefore, the development will also address the issue of the optimum value of K 
for a given value of X. Setting X to be a specified value is interpretable as establishing a feasi¬ 
bility constraint on the search process in RL. We have not constrained A to be greater than 1. 
If A is less than one, less search time is spent in the high target region. 
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The discussion that will follow, then, can be categorized briefly as: 

1. A single expression is developed to be used in obtaining the optimal value of K for a 
given X. 

2. An analytic expression is found for the optimal K value as X approaches zero. 

3. Evolving directly from 2, is an expression for identifying values of M for which the 
optimal K value is infinite for all values of X. (If the optimal K value is infinite then 
the optimal strategy limits search to the high target region.) 

4. Lastly, an expression is found which specifies the values of X, as a function of M, F, 
and /, for which the optimal K is again infinite. 

In this example, the regions are not symmetric and separate differential equations are 
needed for each region. The equations for the expected number of targets in the low target 
region are the same as before, Equations (2) and (4). The differential equations for the high 
target region and their solutions are also basically the same as before except for replacing A, the 
arrival rate, by M x A. We then set up the standard boundary conditions which constrain the 
target level to be continuous and solve for their constants, C t , C 2 , Ci, and C 4 . In this case a 
cycle consists of KX minutes of search in RH followed by X minutes of search in RL. 

The average expected number of targets present in the combined two-region area is then 
determined with the following equation 

(14) 5 - (K lux JT lc > exp< -™ + A/F 

+ C 4 exp( —(F + /)/) + MA/(I + F)\dt 

+ AKl + F ) + C, exp(-/7) + MA/F)dt. 


The first integral represents the average number of targets in both regions during the 
search (KX minutes) of RH and the second, the average during the search of RL. After 
integrating out, substituting where necessary, and combining terms wherever possible, the 
result is 


MA 

F + / F 


AKM - 1) 

(K + 1 )F(F+ I) 


(K + 1)T 


At 2 

fHf + n 2 


M( 1 - exp (-FX)) (I - exp(-(F + l)KX)) 
1 - exp (~(F(K + 1) + IK)X) 

(1 - exp (-FKX)) (1 - exp(-(f ± l)X )) | 

1 -exp(-(F(K + 1) + I)X) 
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Our analysis of this expression is delayed until later; for now we present just a graphical 
analysis. In f igures 2 and 3 we have graphed the optimal value of £ as a function of X, the 
duration ol a visit to RL In each case targets are of a 5 minute duration, F ~ 12. Figure 3 
graphs the relationship lor a range of M (the ratio of high to low region target rates) with the 
ratio / to / set at I A common characteristic of the curves, except for the M equal to one 
cur * e-. is that as A increases, the optimal K value grows asymptotically to infinity (i.e. , search 
only one ugion I Not surprisingly, as M increases the asymptote decreases. As the difference 
in target arrival rates increases, the optimal strategy is more and more likely to focus only on 
the high target region. 


M = 2 1 5 12 



1 2 3 4 X 

I ua k • 2 . the optimal value of A as a function for a range of M values 

R = 4 3 2 1 



1 i a 4 hrs X 

I k.I nr .1 t he optimal value of A as a function of X for a range of values of R 


Figure 3 graphs K as a function of X for a range of R with M set at 1.2. As R increases, 
the cutoff value of X , above which only RH is searched, decreases. For R — 1 the cutoff value 
of A is approximately .43 hours and as R increases to 4 this becomes .18 hours. 
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One characteristic of the curves that seems counterintuitive is that as X increases initially, 
the optimal value of K decreases. A corollary of this is that the optimal stiategy within the 
constraints we have imposed may allocate less than half the search effort to the high target 
region. For identical regions the optimal strategy divides the search equally only in the limit as 
X goes to zero. 


There seem to be two conflicting forces at work. The more intuitive one is that as A 
increases the searcher is forced to incur increased diminishing return in any visit to the low tar¬ 
get region and eventually the cost for visiting the region RL becomes so prohibitive that it no 
longer pays to search the region. This force dominates in the long run. However, there is an 
advantage, in general, to having short cycles and as X increases from 0 there is only one way of 
restraining the increase in cycle length and that is by decreasing K. This propensity for shorter 
cycles seems to dominate the behavior of the optimal value of K for small values of A' turning 
K optimal initially into a decreasing function of X. 

No Constraint on the Duration of a Search 


In this section we focus on the limiting behavior of S' as X approaches zero ti e., no con¬ 
straint on the minimum duration of a visit to a region). Although as X approaches zero the 
optimal solutions are no longer implementable^ the analysis of the limit properties will prove 
significant, because the absolute minimum of S is always approached as X goes to zero. The 
limit of S as X approaches zero is 

/ j / \ I* r = MA . A . AIM __ A I 

t-n l + F F {F + l)(F(K + 1) + IK) F(F(K +"17+77 
The optimal value of K is 


(17) 


K (optimal) =» 


+2R(R 4-1) (M - 1) ± 2(2 R 4- 1) M i,: 
2(R + l) 3 - 2MR : 


where R *= F/1. It is again clear that the optimal value for K does not depend upon the 
independent values of Fand / but only on their ratio, thereby reducing, once again, the number 
of critical parameters. Table 2 displays the optimal K for a range of M and R. For M equal to 
one, which means that the two regions generate targets at the same rate, the optimal value of K 
does not depend upon R but is always one (i.e., equal search in both regions), as was assumed 
earlier in this paper. 


One aspect of the Equation (17) that needs clarification is that since there are two solu¬ 
tions is it possible for both solutions to be positive? As it turns out, this can never happen. 
For the range of M which results in both values of the numerator being positive, the 
corresponding denominator will be negative. If the numerator is negative the denominator is 
positive. This last result provides the key to finding under what conditions the optimal solution 
is to search only one region. 

Search Only One Region 

Whenever both solutions are negative, for M greater than one, the optimal solution will 
be at the upper bound of the feasible region, namely K optimal will be infinite (i.e., search only 
one region). Consequently, the optimal search focuses on only one region whenever 
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TABLE 2 — The OptimaI Value of K as a Function of M and R — No 
Constraint on Minimum Duration of a Visit to RL (X — 0) 


M 

.1 

.5 

1 

2 

5 

10 

20 

30 

40 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1.1 

1.06 

1.10 

1.15 

1.27 

1.71 

3.00 

85.0 

oo 

oo 

1.2 

1.12 

1.20 

1.32 

1.59 

3.01 

45.0 

oo 

oo 

oo 

1.3 

1.17 

1.30 

1.49 

1.97 

6.16 

oo 

oo 

oo 

oo 

1.4 

1.22 

1.40 

1.67 

2.45 

25.0 

oo 

oo 

oo 

oo 

1.5 

1.28 

1.51 

1.87 

3.04 

oo 

oo 

oo 

oo 

oo 

2 

1.52 

2.05 

3.12 

13.1 

oo 

oo 

oo 

oo 

oo 

2.5 

1.74 

2.64 

5.16 

OO 

oo 

oo 

oo 

oo 

oo 

3 

1.95 

3.31 

9.20 

oo 

oo 

oo 

oo 

oo 

oo 

3.5 

2.15 

4.09 

21.2 

OO 

oo 

oo 

oo 

oo 

oo 

4 

2.33 

5.00 

°°. J 

oo 

oo 

oo 

oo 

oo 

oo 


(18) 


M > 


(R + l) 2 
R 2 


To be rigorous, we showed only that in the limit as X approaches zero that the inequality 
specifies whether or not the low target region should be searched. However, if it does not pay 
to ever search the low target region for an infinitesimally short duration there should be no 
incentive to search it if the minimum duration of a visit is instead some number larger than 
zero. 


In order to emphasize the impact of this result we display in Figure 4 the convex region in 
R, M space for which the optimal strategy limits the search to RH. For example, if R, the ratio 
of departure to interception, were 10 then if RH generated 21 percent more targets than RL, 
only RH would be searched. If R is 20, 30, or 40, then if RH generated respectively 10%, 7%, 
or 5% more targets than RL, only the high target region should be searched. The result is a 
second strong limitation on the likelihood of searching more than one region. 

In the preceding analysis, we showed that for given values of Fand /, if the ratio of target 
arrival rates is above some cutoff value, M, the optimal strategy is to search only RH even for 
infinitesimally small values of X. However, even for values of M below the cutoff if X, the 
time to search RL, is ‘too large,' the optimal strategy will again restrict the search to RH. 
Therefore, the next question to resolve is, "When is X too large?” 


In Equation (15) only two components are functions of K, with only the negative com¬ 
ponent a function also of X. The two components are 


(19) 


1 


K + 1 


AI(M - 1) 
F(F + /) 
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Figure 4. A characterization of when to search only high target 
region as a function of two parameters 


0 1 2 S to IS 


K + 1 X XFHF + l) 1 * l 1 
+ 0 - exp(-(F + I)X)) x 


(1 - exp(-(F + l)KX)) 

(i - exp(-(m + i) + nx)) 


(1 - exp (-FKX)) 

(1 - exp(-(F(K + 1) + l)X)) 


If for some value of X the magnitude of the first component were greater than that of the 
second for all values of K, then their sum would always be positive. The optimal strategy then 
would be to allow K to go to infinity, since that would drive each component to zero. 

In comparing their relative magnitudes, we ignore the common factor l/( AT + 1), this 
turns the second expression into a monotonically increasing function of K. Its limit, as K 
grows without bound, is 

A ! 7 

(21) T7- r, 7r - ,o x lAf(l - exp(-F*» + 1 - exp(-(F + /)*)]. 

XF l \F + /r 

The problem therefore reduces to finding values of X such that 


(M - \)F(F + /) 


x [M( 1 - exp(-EO) + 1 - exp(-(F+ /)*)]. 


In Figure 5 we have graphed the lower bound on X for a range of M values with f and / 
set equal to 1. A conservative (sufficient but not necessary) bound on X can be easily found 
since (1 - exp(-FA')! and [1 - exp(-(f + /)A")] are both bounded by i. This bound is 

,23) x * W^\ * 7TF+1) 

which approaches the actual bound as M approaches I. See Figure S for a comparison with the 
actual lower limits. 
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I'Kii Ri 5. Exaci and conservative estimates of the cutoff 
values for X for a range of M. If f is above the cutoff, 
search is limited to the high target region 


6. EXTENSIONS AND SUMMARY 


Multiple Target Types 

In order to inclue the multiple target types in the model, it is necessary to add for each 
target type different values of / and F and a pair of differential equations for each region. To 
calculate the state of the system, S, under a particular search strategy, each of the targets is ini¬ 
tially treated separately. The total average number of targets in progress is then just the sum of 
all the individual averages. The resultant expression is still a function of only X or K and X , 
and can be analyzed in the same manner as before. To illustrate this without going into much 
detail, vte generalize our earlier results which characterize when to search only one region. 


The first expression we derived involves two identical regions with time lost, L , in travel¬ 
ing between the regions. We associate with each target type, j, three parameters, F )y and 
the target specific interception rate, departure rate and arrival rate, respectively. The resultant 
expression for L is now 


I A ' 

/, l 2 

F,(F, + /,) 

Z A ' 

/, 

F,(F, + /,) 


which unlike the earlier expression (13) does depend upon the arrival rates. 


The last two expressions, one for M, the other for Y, relate to the problem of allocating 
search between a high and low target region. When we generalized this problem and allowed 
tor multiple target types, we also indexed the ratio of the two region target arrival rates, M ,, 
and made it target type specific. The expression which corresponds to Equation (18) and 
specifies when to search only the high target region even for small values of X is 


(2 *>) 




V, 

( F, + I ,) 2 



If the above inequality is not satisfied, large values of X will still restrict search to the high 
target region. The conservative form of this equation with multiple target types is 
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( 26 ) 


_ A.IHM. + 1) 

Z -wtT) 

T A,l,iM ,- 1) 

r f }( f , + /,) j 


More Than Two Regions 

Introducing more than two regions into the problem, significantly increases the complexity 
of analysis. Although any of the earlier three inequalities can be applied pairwise to all the 
regions to exclude as many as possible, the complications enter when more than two regions arc 
still left. The heart of the problem lies in the difficulty of defining cycle types to be analyzed. 
The work by Barnett |1] which tackles problems involving more than two regions can yield no 
insight here. In his work any search of a region discovers all targets present with probability 1. 
The consequence of this is that after one unit of search in a region, diminishing return ftom a 
continuous search stops, which is not the case here. 

Summary 

In this paper a differential equation model of a search process was developed which has 
the potential for capturing the dynamics of a sequential cyclic search strategy In displaying the 
application of the model to a number of examples, we have derived a number of independent 
quantifiable constraints which limit the number of regions to be searched and which are sum¬ 
marized in Table 3. The main thrust of all our results strongly emphasize that with randomly 
arriving and departing targets and travel times, search should be limited to <.nk 'he hubstar¬ 
get region. 


TABLE 3 — Summon oj Conditions Under Which 
Only One Region is Searched 
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OPTIMAL STOPPING PROBLEMS FOR DIFFERENTIAL 
EQUATIONS PERTURBED BY A POISSON PROCESS 
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Lincoln Laboratory 
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Lexington. Massachusetts 

ABSTRACT 

We investigate a class of optimal stopping problems for dynamical systems described 
by one-dimensional differential equations with an additive Poisson disturbance. The rate 
of the disturbance may depend upon the current state of the system. A dynamic pro¬ 
gramming equation for the optimal stopping cost is derived along with conditions which 
must be met at the boundary of the optimal slopping set. These boundary conditions 
depend upon whether or not the stopping set may be entered by smooth motion. 


1. INTRODUCTION 

Differential equations driven by Poisson disturbance terms appear in models of the con¬ 
tents of dams and inventories (Prabhu (7]), viral growth (Bartoszynski ll]), population dynam¬ 
ics in random environments (Hanson and Tuckwell [6]), collective risk (Buhlmann [2]), 
machine failure (Wickwire HU), and mass service systems (Prabhu [8]), among others. In 
some of these models the magnitudes of the disturbances are random variables (the disturbance 
term is a compound Poisson process), and in others the disturbances have constant magnitude; 
the latter case is often a useful approximation to the former, and its mathematical treatment via 
the theory of retarded differential equations differs significantly from the more general approach 
taken for distributed magnitudes. Many of these models describe changes in resources or popu¬ 
lations which lend themselves in a natural way to the application of control theory: dam con¬ 
tents, insurance reserves, and inventories must satisfy demands but not be depleted. Queueing 
systems should serve as many customers as possible consistent with running costs; latent 
machine failures ought to be detected early without a large number of false alarms, and malig¬ 
nant viral growth must be checked at acceptable costs for treatment. In this paper we shall 
assume that Poisson disturbances of a one-dimension 'vnamical system are of constant mag¬ 
nitude and shall also suppose that the allowal' >ntr., actiors, at any time, are of a simple 
type: one may either stop the system and gaiti - aich depends upon the current value 

of the system, or let it continue to evolve (in tne ho^ that the reward, at a subsequent time, 
will be greater than its current value). Such problems are called optimal stopping problems and 
their general theory has recently attracted much attention (see Chow, Robbins and Siegmund 
[3] for the case of stopped random sequences and Siryaev [9] for the case of stopped Markov 
sequences and random processes). A treatment of optimal stopping problems which can be 
applied to mixed Markov processes (that is, those whose sample paths have jumps superim¬ 
posed on continuous movement) has been given by Grigelionis (51, but his most important 
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result, the derivation of conditions satisfied by the payoff function at an optimal sloping boun¬ 
dary, does not, apparently, include the case of a process which can cross such a boundary only 
by means of a jump. We shall derive here a set of conditions satisfied by the payoff and 
optimal stopping boundary which applies to both smooth and discontinuous entry into the set of 
states at which stopping is optimal. We shall limit ourselves to autonomous dynamics defined 
on a finite interval, because this case exhibits the essential features of a wider class of prob¬ 
lems. We remark that optimality conditions for problems with control actions more general 
than those considered here—actions which alter paths without stopping them—may often be 
derived in an analogous way. 

2. FORMULATION OF THE STOPPING PROBLEM 


Let U,), ( ^ 0, be a completely observable Markov process on the probability space 
( E. S'. P 1 , where £ C R { . Since we shall consider £ in what follows to be a finite interval, no 
generality is lost by taking £ — 10, f ], f < °°, and we shall do so. Suppose that x, satisfies the 
stochastic differential equation (Gihman and Skorohod [4]) 

(1) dx, = aix,)dt + f ,/3(x,. u)N(dt. du ). x 0 = x. 

where a (•) is continuous on £ and nonzero on £ — ({0} U |£|). (The arguments below can be 
modified without difficulty to account for functions a if which vanish inside £.) The function 
j3 (x. w), the magnitude of the disturbance, is defined as 


( 2 ) 


. € if » > X (.v) 

Pix. u) = 

0 otnerwise. 


where |e i is a bounded constant and X(x) > X 0 > 0 is >he rate al which disturbances occur. 
/V(r, •) is a random Poisson measure in R' such that for any Borel set A C R\ E Nit, A) - 
tniA). where Tris a finite measure which assigns mass X(x) to the interval [X~'(x), °°1. Solu¬ 
tions of Equation (1) satisfy the ordinary differential equation dx/dt - a(x) between distur¬ 
bances of magnitude «, and the latter occur according to a Poisson process with rate X (x,). The 
infinitesimal generator A of (x,|, which acts on bounded, continuous functions f. E — /?’, is 
defined by Afix) » aix)f'ix) + X (x) l/(x + e) - /(x)j. The behavior of a(x) and the sign 
of e will determine the boundary conditions which may be imposed at x = 0 or x = { upon 
solutions of Afix) = bfix ), an equation which will occur below. 


Let gif ^ 0 be a bounded, continuous function on R l , and M the set of stopping times 
relative to the family of o - algebras S' C ^generated by {x,|. The optimal stopping problem 
we shall consider is to find a stopping time r 6 M (if one exists) such that for any x € £ 

(3) E x e~ br gix-) = sup E x e~ bT g(x T ) = s(x), b ^ 0, 

7 T(* 

where E x denotes the expectation conditional on x 0 = x. In the next section we shall derive a 
dynamic programming equation satisfied by the payoff s(x). 


3. A FUNCTIONAL EQUATION SATISFIED BY six) 


Suppose that x 0 * x and we wish to follow an optimal stopping policy for ail future time. 
If we stop the process, we gain gix) as payoff; if we decide to wait a time A > 0 for a higher 
payoff, our expected gain is E x e~ bA s(x + Ax), where Ax is the distance x, has moved during 
A. Since we may only wait or stop the process, we have 
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(4) s(x) = max (g(x). E x s(x + Ax)e~ b *\ 

and if s(x) 6 D A> the domain of A , E x e hX six + Ax) = s(x) + AAsix) - hAs(x) + o(A), 
whence 


six) = max {g(x). six) + AAs(x) - bAs(x) + o(A)|, 
or 

(5) 0 = max |g(x) - s(x). A(/Is(x) - fc(x)) + o(A)|. 

Since A > 0 is arbitrary, we have the dynamic programming equation 

(6) max {g(x) - six). 4s(x) - bsix) 1 = 0, s € D A . 

If y is the set of points in E at which it is optimal to stop, and “<f - E - if is the optimal con¬ 
tinuation set, we have from Equation (6) that 

(7) s(x) = g(x) and Asix) < bsix) <=—> x 6 if 

and Asix) — bs (x) and six) > g(x) <=*=> x € <g . 

By Theorem 6, p. 108, of Siryaev [9], f, the optimal stopping time, is equal to inf 
(/ ^ 0: x, €y), provided that P(i < °o) = 1. The equations at (7) are not enough to deter¬ 
mine an optimal policy (and therefore six)) because they do not specify the boundary of if 
The next section is devoted to a derivation of condtions for "matching” six) to g(x) at the 
boundary of if. These conditions will serve to determine an optimal policy uniquely in most 
cases of practical interest. 

4, OPTIMAL MATCHING CONDITIONS AT bif 

The nature of these conditions depends upon whether x, crosses bif oy smooth or discon¬ 
tinuous movement. The following theorem treats the case of continuous movement and is part 
of the lore of optimal stopping for various continuous processes. It says, in our case, that if x, 
can enter yby smooth movement, the payoff six) will be a smooth function on bif inside E. 

THEOREM 1: Let x 0 =<r € b.f <r & 0, £, and suppose that for any small A > 0, 
x^ € ^ if there is no jump during [0, Al. Assume further that D(x) = six) - g(x) is in D A 
at x — cr, s’(<r) and g'icr) exist and are continuous with the derivatives taken for x —*o- from 
within ‘ST. Then 

/01 dsiv) dgi<r) 

(8) ~dT~ ~dT’ 

where dldx is the left (right) derivative if aic) > (<) 0. 

PROOF: (A modification of an argument first used by Grigelionis [5]). By hypothesis, 
E„Dix±) — A A Dix) + oi A) for small A > 0. On the other hand, 

E„ Dix±) = Dicr + ai<r)A) |l - J \(m(s))ds| 

+ Dicr + «) f A imis))ds + o(A) 

Jo 

• Di<r + «) A(o-)A + oiA), 
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where dm(s)/ds=‘ a(m(s)), m(0) = cr. Hence, 

AD (cr) = A (cr) D (cr + t ), 
or 

a(cr)D'(cr) 4- k(cr)D(cr + e) = A(cr)Z)(cr 4- e), 
so that (a (<r) ^ 0) 

s'(cr) = g'(cr). 
which was to be proved. 

The following theorem treats the case in which x, can enter ./only by means of a jump. 
Grigelionis’ theorem does not apply here because the expected exit time from a neighborhood 
of cr into /is bounded away from zero. The proof of his theorem also requires that s be in the 
domain of A at a-; since solutions of A/(x) = bf(x ), x € /(x) = g(x), x 6 /, generally 

have a discontinuity at cr in this case, it is not a priori clear that s satisfies this requirement. 
We shall see that the requirement for matching of derivatives is replaced by that of continuity 
of s across 8/when x, can enter /only by a jump. 

THEOREM 2: Suppose that x 0 = cr € 8/ cr ^ 0, f, and that for any small A > 0, 
x A € <£ if x, has no jump during {0, A], Then the following conclusions hold: 

(a) if cr + € € # and o- - e € / then no optimal boundary condition may be assigned at 

cr; 

(b) if o- + e € /, then 

(9) s(cr±) = g(cr) if e $ 0, 

where s(cr ±) = lim s(x). 

x— 1 V 

PROOF: Under the conditions of (a), cr is not accessible on an optimal path from < €. 
Consider (b). By the definition of s, s(cr ±) > g(x). Now suppose that instead of stopping at 
o- (which is optimal) we let x, evolve for a short time A > 0 during which x, moves a distance 
Ax. Then 

g (cr ) > E a s (cr + Ax)r‘“ 

* (s (cr ± ) + a (cr )A s '(cr) ] (1 — A (cr )A) (1 — b A) + s (cr + «)A (cr ) A+ o (A) 

“ A (a (cr )s'(cr ) + A (cr )s(cr + e) — (A (cr) + i)s(cr ±)] + j(cr ±) + a (A), 

where s' is the appropriate one-sided derivative. Since A > 0 is arbitrary, we have 
s (cr ±) < g(cr), whence the theorem is proved. 

5. AN EXAMPLE 

Suppose that a (x) - a, A (x) — A > 0, g(x) - x 2 and that 0 and f are reflecting barriers 
if they can be reached by x,. This is a simple model for a dam or inventory whose input (out¬ 
put) is described by the ordinary differential equation dm(s)/ds - a, and whose output (input) 
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is described by a Poisson process with rate A and jumps of size e. The reflecting barrier 
assumption implies that the content of the corresponding system stays at 0 (or £) if it reaches 
or jumps over this level until new input (or output) causes the content to assume a value in 
(0, £). Then by Section 3 

six) — sup E x e br x 2 


satisfies 


( 10 ) 


,4s (x) = bsix) in V 
s(x) = x 2 in y 


where /4s(x) = as'ix) + X (s(jc + e) - s(x)]. Using the fact that hix) = E x e b *g(xJ is 
convex if g is convex, one can show, via Equation (4), that for this example the optimal stop¬ 
ping set must be of the form i/ = [cr, for some cr € [0, £]. Conditions which determine 
the optimal value cr of a depend upon the signs of a and e. 


CASE l: a > 0, e > 0. Here cf + e € .^and s(cf +«)= gtf + e). Applying Theorem 
l and taking Equation (10) into account, we find that if cf € (0, £), then it must satisfy 

(11) 2aor + A((cr + «) 2 — cr 2 ] — b<r 2 = 0, 

a quadratic in cr Only the root 

or + = [a + A« + yJia +Ac) ! + bkt 2 ]/b 
can be cf and one has cf — min (cr + . £ ). 


CASE 2: a > 0, e < 0. In this casecf +« 6 if and Theorem 1 yields 
(12) 2acr + As(cr - |«|) - (A + b)cr 2 = 0 


or s (cr — |e |) = o- [(X + b )<r — 2a 1/A 

as an equation which must be satisfied by <f if it lies in (0. £). Since s(cr — |e |) ^ gi<r — 
|e |) = (cr - |e |) 2 , Eq uation (12) implies t hat cf must satisfy bcr 2 - 2 (a — A |e |kr — A« 2 ^ 0, 
or cf > [a - A |c| + \f ia — A |« |) 2 + bkt 2 ]/b. To determine cf from Equation (12), we note 
that solutions of the functional equation Afix) = bfix), x € (0, cr); fix) = 0, x < 0; 
/(cr) = cr 2 , /'(cr) = 2cr, will have a discontinuity of magnitude / 0 = /(0+) > 0 at x — 0, 
which is determined to satisfy the boundary condition /(cr) — /(cr, /o)“<z 2 . The optimal 
value of cr is then chosen to fulfill /’(cr) — 2cr or /(cr — |«|) — cr[(A + b)a — 2a]/k (cf. 
Equation (12)). 

CASE 3: a < 0, « < 0. Here x, is decreasing with probability one and we clearly have 
y- E and six) ~ x 2 for all x € £. 

CASE 4: o < 0, « > 0. Theorem 2 applies to yield s(cf-) = cf 2 as the optimality condi¬ 
tion for cf, Since x — 0 is reflecting, s'(0) — 0, or s(t+) = (1 + f>/A)s(0-L). A procedure for 
determining cf is then the following, for cr € (0, ?), solve 4/(x) — bfix), x € (0, a), 
/(x) — x 2 , x € (cr. |1, fix) sf, x > £. Solutions of this equation will generally have a 
discontinuity of magnitude f„ = /(cr-) at x - cr, which must be chosen for this cr to satisfy 
the reflecting boundary condition at x — 0. Finally, cf is chosen such that the optimality condi¬ 
tion /(cf-) - cf 2 is also met. Note that the resulting function fix) (« six)) will then be con¬ 
tinuous on (0, £) and will also have a continuous derivative there. 
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6. CONCLUDING REMARKS 

i. If aix) = 0 in (0, f) or A(x) is not bounded away from zero, x, may not attain a stop¬ 
ping set with probability one, and stopping policies must generally include prescriptions of what 
actions to take if x, enters a set in E which is absorbing. 

ii. Cost functionals other than (3) may sometimes be treated within the present frame¬ 
work. The costs 

£ x J* o c ( x t ) ds. E x J#(x r ) — c(Xj)ds| 
and 

| A + £ 0 J* Q c(x s )ds|/£or 

and their discounted versions are such functionals. For example, in the second case, if 
P(t < °°) = 1, then an equivalent problem is to find an optimal stopping lime for the cost 

(13) fix ) + E x F(x t ). 

where F(x) = j?(x) - /( x) and fix) satisfies A fix) - c(x)= 0 (cf. Taylor [10]). Nonauto- 
nomous costs or dynamics, or both, may also be treated by augmenting x, with time as a second 
dimension. 

iii. If |e | « 1, machine solution of the retarded or advanced differential equation 
A fix) - hfix) - 0 may cause numerical difficulties, especially if there are singularities caused 
by zeros of u(x). Singular perturbation expansions of fix) in the form £ €'£,(0), where 0 is 

i>0 

a suitable stretching transformation (e g., 0 = x/«), and the £, are to be found, may be used in 
this case to derive accurate approximations to s(x) and 
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DECOMPOSITION ALGORITHMS FOR MINIMAL CUT PROBLEMS 
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ABSTRACT 

Consider a network GiN. A) with n nodes, where node 1 designates_its 
source node and node n designates its sink node. The cuts (Z,. Z ( ), 

/-I. n - I are called one-node cuts if t € Z,. n 9 Z,, Z| - ll), 

Z, C Z, +l and Z, and Z /+1 differ by only one node. It is shown that these 
one-node cuts decompose G into I ^ m < nil subnetworks with known 
minimal cuts. Under certain circumstances, the proposed one-node decomposi¬ 
tion can produce a minimal cut for G in 0 in 1 ) machine operations. It is also 
shown that, under certain conditions, one-node cuts produce no decomposition. 

An alternative procedure is also introduced to overcome this situation. It is 
shown that this alternative procedure has the computational complexity of 
0(n 3 ). 

1. INTRODUCTION 

Determining the location of minimal cuts in a flow network is gaining wider importance in 
recent literature. Jarvis {8, 9] shows that in a communication network, if each arc is assigned a 
value which represents the effort of disabling that communication link, then the optimal stra¬ 
tegy to disable the communication between the message sender and the message receiver is to 
destroy the links (arcs) on a minimal cut. Phillips and Dessouky [16] indicate that the 
time/cost tradeoff problem can be converted into a problem of maximal flow where at each 
iteration the activities on the minimal cut are crashed. Rhys [18] and Pickard [17] indicate that 
a selection problem of shared fixed costs can be converted into a maximal flow problem where 
minimum cut identifies the optimum options selection. 

Some efTort has been made to find a minimal cut or all the minimal cuts in a directed net¬ 
work, regardless of maximal flow (see [7, IS]). However, all the labeling algorithms for finding 
the maximal flow provide the location of a minimal cut as a by-product at the end of the algo¬ 
rithms (3, 4, 61. The maximal flow algorithms proposed by Dinic [2] and Karzanov [12], find 
the maximal flow but they do not provide a minimal cut. However, finding the location of a 
minimal cut in these algorithms can easily be achieved by applying a labeling procedure on the 
network once the maximal flow is obtained. 

Among the efficient labeling algorithms, the first-labeled first-scanned algorithm proposed 
by Edmonds and Karp [3] has a theoretical upper bound of (n 3 - n)/4 augmentations, where n 
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is the number of nodes in the flow network. Among the efficient nonlabeling algorithms, 
Karzanov’s [12] algorithm with preflows has the theoretical upper bound of 0(n 3 ) machine 
operations. 

in this paper we propose two decomposition algorithms for locating a minimal cut in a 
given flow network. The network is decomposed into 1 < m < — subnetworks with known 

minimal cuts by utilizing a one-node cut concept. Section 2 gives the decomposition of the net¬ 
work into two subnetworks by an arbitrary cut. The relationship between the minimal cuts of 
the subnetworks and the arbitrary cut is given in Theorem 1. The concept of one-node cuts 
and some related theorems are introduced in Section 3. Section 4 presents an algorithm for 
decomposing the original network into three subnetworks where the intermediate network con¬ 
tains a minimal cut. Finally, conclusions are given in Section S. 

2. MINIMAL CUTS AND NETWORK DECOMPOSITION 


A network G(N. A) is a collection of nodes and ordered pairs of nodes which are called 
arcs. N designates the node set and A designates the arc set of the network. 


There are two special nodes in an n-node flow network; one is called the source node, 
denoted by 1 (or s), and the other one is called the sink node and is denoted by rt (or f). 


With every arc (/, j) € A of G we associate a positive integer m /; , called the capacity of 
the arc. A set of nonnegative integers f tj is called a feasible flow in a network if they satisfy 
the following equations: 


fij T*. /,(, 


-/ if y - 1 
0 if j * 1, n 
/if J-n 


and 

0 < / y < u,j for all (/. j) € A. 

where / is a nonnegative integer called the value of the flow. A set of feasible flow, ffj, which 
maximizes / is called a maximal flow. 


DEFINITION. Let A" be a subset of the nodes in G such that_l € A , n Q X. Also let 
X - N - X. Let {X, AT be a set such that the arc (/', j) € (A\ AT if / € X, j € X and 
(/. j) € A or j € X, i € X and (/, j) € A. Such a set is called a cut (or a cut-set). 

The capacity of a cut (X, X), denoted by c(X. X), is given by 
c(X, X)- X V 

( i.iH U.A t 
i€ X./( X 


DEFINITION: A cut with minimum capacity in a network is called a minimal cut (min- 

cut). 
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DEFINITION: A cut (X. X) is said to lie on the left of a cut ( Y. Y) if X Q Y. Con¬ 
versely, a cut ( Y. Y) is said to lie on the right of a cut (X. X) if Y 2 X. 

DEFINITION: A cut ( X. X) is said to lie between two cuts (Y, Y) and (Z, Z) with 
Y C Z, if Y C X Q Z. 

DEFINITION: Two cuts ( X , ^)_and _( Y, Y) are said to cross each other if each one of 
the sets TO Y, X D Y, X D Y , and X D Y is nonempty. 

Ford and Fulkerson 14] show that for single commodity network flow problems the value 
of a maximal flow, /*, is equal to the value of the capacity of any minimal cut in that network. 

Let G — (N, A) be a network with jV — {1. rt}. Also, let ( X, X) be an arbitrary 

cut. Let all the arcs (/, X) € ( X, X) be collected to an artificial node s with 

u is — £ «„ for each / € X. 

ax 

Similarly, let the arcs (X. j) € (T, X) emanate from s with 
u sj - £ u„ for each j € X. 

i(X 

Note that the arcs (J, i) € ( X, X) are ignored in this procedure (since they have no effect on 
cut values). 

By this procedure we create two subnetworks connected to each other via a single artificial 
node s. Le*. G, = (X U {j}, A\) be the first subnetwork where the arc set A x contains all the 
arcs (/. j) € A and i € X, j € X plus the arcs 0, s), i € X as described above. Similarly, let 
G 2 “_(|s} U_ X, A 2 ) be the other subnetwork, where A 2 contains all the arcs (/, j) 6 A and 
/ € X^j € X as well as the new arcs (s, j), j € X as described above. Note that c(X, s) = 
c(s, X) - c(X, X). Also note that the artificial node s serves as a sink node for G, and as a 
source node for G 2 . 

THEOREM 1: Let^a network G( N, A) be partitioned into two subnetworks G\ and G 2 
by an arbitrary cut ( X, X), as described above. Suppose that (Z\, Z,) is a minimal cut for G, 
and (Z 2 , Z 2 ) is a minimal cut for G 2 . Then the following statements are true: 

(a) max{c(Z,. Z,), c(Z 2 . Z 2 )! ^ c(X, X ), 

(b) there exists a minimal cut for G which lies between (Zj, Z,) and (Z 2 , Z 2 ), 

(c) if c(Z|, Z|) - c(X, X) > c(Z 2 , Z 2 ), then (Z 2 , Z 2 ) is a minimal cut for G, 

(d) if c(Z 2 , Z 2 ) - c(X, X) > c(Z|, Z|), then (Z h Z\) is a minimal cut for G, 

(e) if c(Z|. Z,) - c(Z 2 , Z 2 ) - c(X, X ), then the cuts (Z,. Z,), (Z 2> Z 2 ) and (AT, X) 

are all minimal cuts for G. 
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PROOF: 


(a) If c(Z), Z|) > c(A, A - ), thenjve would have chosen (A, A) as the minimal cut 
between 1 and s. Thus, c(Z(, Z t ) < c(A, A). With the same line of argument we 
have c(Z 2 . Z 2 ) < c(A. A). 

(b) Suppose that O', Y) is a minimal cut for G which crosses (Z t , Z\) and (Z 2 , Z 2 ) as 
shown in Figure 1 below. 


0 


(X, X) (Z 2 , z 2 ) (Y. Y) 

Figure 1. A minimal cut (K, Y) crossing (Z,. Z,) and (Z 2 . Z 2 ). 

Consider the following node-sets: 

a - ?_n z, n z 2 , fl_- fn z ( n z 2 , c - ?n z, n Z;, d= r n z, n z 2 , £- 
K n Z, n z 2 , £- Y O Z, n Z^ Note that ^UU«Z,, CUDU£U£«Z, 
,4UfiUCUD-Z 2 , £U£-Z 2 , RUOUF- Kand /< U C U £ - K 

Consider the cut (5, iV - £). Since £ C Z\ this cut lies completely in G] and thus 
c(B, B) > c(Z x , Z,) 

where N - B ~ A U C U DU E U F. Thus, we have 

0) c(B, A) + c(B, C) + c(B, D) + c(B, E) + c(B, F ) ^ c(A. C ) + c(A, D) 

+ c (A, E) + c(A, F) + c(B. C) + c(B, D) + c(B, £) + c(B, F). 

Similarly, consider the cuts (N — £, £) and (Z 2 , Z 2 ) which lie in the subnetwork G 2 . 
With the same line of argument we have 

(2) cM, £) + c(B, £) + c(C, £) + c(D, £) + c(£, £) > cM. £) + c(S, £) 

+ c(C, £) + c(D, E ) + c(4, £) + c(B, F) + c(C. F) + c(D, F). 
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Adding (1) and (2) together and subtracting c(B, E ), c(C, E ), c(B, D), c(A, E) and 
c(B, F) from both sides we get 

(3) c(B, A) + c(B, C) + c(B, E) + c(D, E) + c(F. £) > cU, C ) + c(A, D) 

+ c(A, E) + 2 c(A. F) + c(B, C ) + c(B, E) + c(B, F) + c{D, E) 

+ c(C, F) + c(D, F). 

We also have 

(4) 0 >-c(A, D)-c(A, F)~ c(C. F), 

(5) c(D, C) - c(D. C ) 
and 

(6) c(£>, A) + c(F. A) + c{F, C) > 0. 

By adding (3), (4), (5) and (6) we get 

(7) c(B. A) + c(B. C) + c(B, £) + c(D, £) + c(£. £) + c(D, C) + c(0. /D v ' 

+ c(F, A) + c(F, C) > c(A, C ) + c(A. E) + c(A, F) + c(B. C) 

+ c(B, E) + c(B , F) + c(D, C) + c(D, £) + c(D. F). 

The left-hand side of (7) is the capacity of the minimal cut ( Y, Y ) and the right-hand side is 
the capacity of the cut (A U B U D, C U £ U £). Thus, 

(8) c(Y. Y) > cC4 U B U D, C U £ U £). 

The inequality in (8) can only be satisfied as an equality. Therefore, we conclude that the cut 
(A u B U D, C U E U F) is also a minimal cut for C. 

(c) Since c(Z t , Z]) - c(Af, X) both are minimal cuts for G t . From the inequality (8) 
in part (b) we have shown that there exists a minimal cut for G which does not cross 
a given^ minimal cut for G\. Since (X, X) is a minimal cut for G, and since 
c(Z 2 , Z 2 ) < c{X, X) then (Z 2 , Z 2 ) is a minimal cut for G. 

(d) With the samejine of argument as in part (c), we have (X, X) as a minima) cut for 
G 2 andc(Z|, Z\) < c{X, X). Therefore, (Z\ , Z\) is a minimal cut for G. 

(e) Since c(Z t , Zj) — c(X, X)- c(Z 2 , Z 2 ), by considering (T, 30 as a minimal cut 
for G\ as in part (c), we conclude that we can Jind a minimal cut for G which does 
not cross (A’, 30. Therefore, (Z 2 , Z\), (Z 2 , Z 2 ) and (3f, X) are all minimal cuts 
for G. 

Consider subnetworks G t and G 2 . Since both are legitimate networks with a source and a 
sink node, they can similarly be partitioned into two subnetworks Gn, Gi 2 , and G 2 i, G 22 , 
respectjvely, by arbitrary cuts {X t . X\) and (X 2 . X 2 ) where X t C X and X C 3f 2 . JLet 
(Z||, Z(|) and (Z l2 . Z, 2 ) be minimal cuts for G 21 and G) 2 , and let (Z 2 i. Z 21 ) and (Z 22 , Z 22 ) 
be minimal cuts for G 2t and G 22 , respectively. This situation is depicted in the figure given 
below where $| is the super node between Gu and G )2 , s 2 is the super node between G 2l and 
G 22 and s is the super node between Gj and G 2 . 
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Figure 2. G\ and </ 2 are further decomposed into G M . <7 12 , and G 2I . G 22 . 


Therefore, we can hopeto find a minimal cut (Z t , Z ( ) for G! by considering (X\, X\) 
and the minimal cuts (Zjj. Z u ) and (Z, 2> Z 12 ) for subnetworks G n and G 12 , respectively. 
This process leads us to the idea of ultimate decomposition which we will call one-node decom¬ 
position. If this partitioning of subnetworks is repeated iteratively, X, and X i+i will eventually 
differ from each other by one node. The following section introduces the one-node cuts and 
some important property. 

3. ONE-NODE CUTS 

Let a network G(N, A) contain n nodes. Also, let the nodes of the network be num¬ 
bered from 1 to n, where node 1 is the source node and node n is the sink node. 

Consider the following noncrossing cuts: 

(Z h Z,), i — 1, 2. (n — 1), where 

Z, ” U O'}. Z ( - N - Zj. 

Thus, Z,- (1), Z 2 - {1. 2}.Z„_,- {1, 2 . n - U, 

and Zj — {2, 3.«}, Z 2 - (3, 4.n). Z B _j —{«). 

We will call these cuts one-node cuts since Z, differs from Z, +l by one node. The values of the 
cuts, c(Z f , Z,), / - 1. n - 1 viewed as a sequence of numbers, will produce subse¬ 

quences of increasing and/or decreasing values. 

Graphically, we will have a picture similar to the one given in Figure 3 below. In this pic¬ 
ture the downward arrows represent the decreasing subsequences and upward arrows represent 
the increasing subsequences. 
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Figure 3. A typical picture of one-node cuts and their relative values. 


THEOREM 2: If cCZ,. Z,) > c(Z i+l . Z j+1 ) for/- 1. 2. n - 2, then (Z„_,. Z„_,) 

is a minimal cut for G. 

PROOF: Let ( Y. Y) be a minimal cut for G which lies on the left of Z„_|). Con¬ 

sider the node set Y. We can find an index k such that Z, Q Y for j ^ k and Zj £ F for 
j > k. (Note that Z x Q Y.) Let Figure 4 depict this situation. 

(Z k . Z k ) 
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Let Z* +1 n Y - M, Z* +1 n Y - {* + 1} - N, Z* + , D Y - L and Z* +1 n Y - T. 

We have 

2*) ^ c(Z* + i, Z* +1 ) 
or 

c(W, N U L U T) ^ c(M U N, L U T) 
or 

c(M, N) + c(M, L ) + c(Af D Ss c(Af L) + c(M, T) + c(N. L) + c(N. D. 

Since 

0 < c(N. L) 

we can write 

(9) c(M. AO + c(M, T) > c(/V, D + c(M, T). 

Since 

(10) c(L. T)-c(L. T) 
and 

(11) c(L, /V) > 0, 

by adding (9), (10), and (11), we get 

(12) cU. T) + cU, N) + c(M, T) + c(Af, N)>c(N, T) + c(M, T) + c(L. T). 

The left-hand side of (12) is nothing but c(Y, Y) and the right-hand side is c(Y U [k + 1), 
Y— [k + l|). Therefore, 

(13) c{Y, Y) > c(Y U {* + 1). Y- [k + 1}). 

Hence, the cut O' U {k + 1}, Y — [k + 1}) is as good a cut as O', Y). By assuming the new 
cut O' U (A: + It, Y — (k 1)) as a minimal cut for G we can apply the same reasoning again. 
Thus, we will eventually reach the conclusion that (Z„_Z B _j) is a minimal cut for G. 

THEOREM 3: If c(Z,, Z,) < c(Z m , Z i+I ), / - 1.(n - 2), then (Z,. Z,) is a 

minimal cut for G. 

PROOF: The proof of this case is very similar to that of Theorem 2. In this case we find 
an index k such that Z, 2 Y for j > k and Z t 2 Y for j < k. This situation is depicted in Fig¬ 
ure 5 below. 

We define M - Z*_, O F, N - Z*_, n Y, L - Z*_, n ?, and T — Z*_, n ?. 

We have 

c(Z*_|, Z*_|) ^ c(Z k , Z k ) 
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(Y, Y) 



or 

c(M U L, N U T) < c(M U L U N. T) 
or 

c(M. y) + c(M. T) + c(L. N ) + c(L, T) < r(M T) + c(L. T ) + c(N. T). 

Since c(L. N) ^ 0, 

we have 

(14) c(M. N) + c(M. T) < c(Af, T) + c(N, T) 

and 

(15) c(M. L)- c(M, L) 

and 

(16) 0 < c(N, L). 

By adding (14), (15), and (16), we get 

c(M, N ) + c(M, T) + c(M, L) < c(M, T ) + c(N, T ) + c(M, L) + c(N, L) 
or 

c(Y — (*|. Y U { k }) < c(Y, Y). 

As in Theorem 2 by rede *"'ning_ the minimal cut ( Y, Y) and applying the same argument, we 
reach the conclusion that t ?|, Z\) is a minimal cut for G. 
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THEOREM 4: If the one-node cuts produce only a decreasing sequence followed 
immediately by an increasing sequence, then the one-node cut with the smallest capacity is a 
minimal cut for G. 

PROOF: Let p be the index where c(Z,, Z,) > c(.Z i+lu Z i+l ) for 1 $ /' < (p — 1) and 
f(Z,, Zj) < c(Z l+ |, Z i+ 1 ) for p < / < (n — 2). Let (Z p , Zp)_divide the_network into two 
subnetworks G ( and G 2 as described before. (Note that (Z p , Z p ) is (X, X) in Theorem 1.) 
The subnetwork Gy contains only a decreasing sequence and the subnetwork G 2 contains only 
an increasing sequence. From Theorem 2, ( Z p , Z p ) is a minimal cut for Gy and from Theorem 
3, (Z p , Zp) is a minimal cut for G 2 . Then from Theorem 1 we conclude that ( Z p , Z p ) is a 
minimal cut for G. 

THEOREM 5: If the one-node cuts produce only an increasing sequence followed by a 
decreasing sequence, then either the one-node cut with minimal capacity is a minimal cut for 
G, or else there exists a minimal cut for G which crosses the one-node cut with the largest cap- 
city. 


PROOF: Let p be an index such that c(Z,, Z t ) < c(Z ;+ |, _Z i+1 ) for 1 < / < (p — 1) 
and c(Z f . Z,) > c(Z l+ |, Z /+l ) for p < / < (n - 2). Let (Z p , Z p ) decompose G into two 
subnetworks G, and G 2 as described before. From Theorem 3 we get (Z\, Z\ ) as a minimal 
cut for Gi and from Theorem 2 we get (Z„_ ( , Z„_,) as a minima) cut for G 2 . Therefore, from 
Theorem 1-b we conclude that there exists a minimal cut for G which lies between (Z,, Z,) 
and (Z„_|, Z„_j)- If this minimal cut is not either (Z h Z\) or (Z„_j, Z,_)), then it cannot lie 
completely in one subnetwork. Hence, it must cross the cut ( Z p , Z p ) which separates G\ and 
G 2 . 

COROLLARY 1: For any network G(N, A), the one-node cuts decompose G into 
1 < m < («/2) subnetworks with known minimal cuts. These subnetworks are characterized 
as the collection of nodes such that the one-node cuts corresponding to these nodes form a 
subsequence of consecutive decreasing and increasing values. If the first subsequence is an 
increasing one then the nodes corresponding to this subsequence constitute the first subnet¬ 
work. If the first subsequence is a decreasing one then the nodes corresponding to the first 
decreasing and increasing subsequences constitute the first subnetwork. After the first subnet¬ 
work Gy is determined, the remaining subnetworks G /( i - 2. m can be established by 

considering the nodes in each consecutive decreasing and increasing subsequences. If the one- 
node cuts produce a single sequence of increasing or decreasing values, then a minimal cut for 
G can be found by choosing the smallest capacity one-node cut. 

A minimal cut for each subnetwork is given by the one-node cut in each subnetwork with 
minimal capacity. (Note that the last subnetwork may contain only a decreasing sequence.) 

The computational complexity of decomposition by one-node cuts can easily be given as 
(n - 1) J additions and/or subtractions and (» - 2) comparisons. 

Therefore, the one-node cuts can produce a minimal cut for a network in 0(n 2 ) additions 
and/or subtractions if the one-node cuts produce a single subnetwork (cases in Theorems 2, 3, 
and 4). 
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It is clearly obvious that it is to our benefit to end up with a single subnetwork (the net¬ 
work itself) as the result of one-node cuts. The following section exploits this situation. In the 
approach presented in the next section, the one-node cuts are selected in such a way that either 
the one-node cuts will produce a single subnetwork or we hope that some nodes will be elim¬ 
inated from the original network so that a smaller subnetwork can be solved for a minimal cut. 

4. THREE-DECOMPOSITION PROCEDURE 

The name three-decomposition stems from the fact that we would like to decompose a 
given network G into three subnetworks G t , G 2 , and Gj with the property that a minimal cut 
for G can be found by only considering the intermediate subnetwork G 2 . 

In order to achieve this goal we will try to obtain a decreasing subsequence of one-node 
cuts starting from node 1. If this process does not exhaust all the nodes of G then we will try 
to obtain an increasing subsequence of one-node cuts ending at node n. This process will pro¬ 
duce one of the following three outcomes: 

(a) c(Z h Z,) $5 c(Z j+1 , Z 1+l ). /- 1. n - 2 

(b) c(Z,. Z.,) < c(Z,+|. Z /+1 ) , /- 1. 2. n - 2 

(c) c(Z,, Zj) > c(Z, + t . Z 1+1 ), /—I.(k - 1) and 

c(Z,_|, Z,_.) < c(Z,, Z,), i « (n - 1), (n - 2). p + 1 

and k < p. 

If the outcome is as given in case (a) then from Theorem 2, (Z„_), Z^_|) is a minimal cut for 

G. If the outcome is as given in case (b) then from Theorem 3, (Zj, Z\) is a minimal cut for 

G. We have three possible subcases for an outcome described in case (c): 

(i) k — p. In this subcase the network contains a decreasing and an increasing block 
and, hence, (Z k . Z k ) is a minimal cut for G. 

(ii) 2 < pk < 3. In^ this case (Z*. Z k ) is a minimal cut for G if c(Z k ,_Z k ) < 

c(Z„, Zj,) or (Z p , Zp) is a minimal cut for G if c(Z p . Z p ) < c(Z k , Z k ). If 

c(Z*. Z k ) = c(Z p , Z p ), then both cuts are minimal cuts for G. 

(iii) p - k ^ 4. In this case either the one-node cut with minimal capacity is a minimal 
cut for G or else there exists a minimal cut which lies between ( Z k , Z k ) and 
( Zp . Z p ). 

The following algorithm formalizes the foregoing discussion. 

ALGORITHM: 

Step 0. Let Z, - {l), Z, = N - (l}. Also let c(Z,, Z,) •= £ u Xj , and k = 1. 

<i.y7€,4 


VOL. 28, NO. 3. SEPTEMBER 1981 


NAVAL RESEARCH LOGISTICS QUARTERLY 











442 


S TUEEKCI 


Step 1. Determine 

c(Z k U {y*|, Z k - {j*}) - min (c(Z* U (>}. Z k - { 7 })). 

Z k 

i*n 

U c(Z k , Z. k ) > c(Z k U 0*1, Z k - {y*}), then set Z* + 1 - Z k U 0*1 and 
Z *+1 ** N — Z* +1 , k - k + 1 , go to Step 1. Otherwise, go to Step 2 . 

Step 2. Let Z„ H - {«}. Z„_, « N - Z„_,. Also let 

r(Z n _i, Z„_,)» £ u m , and p ** n - 1. 

0.»K4 

Step 3. If p = k , go to Step 4; otherwise determine 

c(Z p ~ 0*1. Z p U 0*1) = min [c(Z p - 0). Z p U (y})|. 

J€Z p 

j* z k 

If c(Z,. Zp) > c(Z p -0*1, Zp U 0*1). then let Z,_, = Z p - 0*1, Z„_, = 

/v ^,-h and p — p — 1 , go to Step 3. Otherwise, go to Step 4 . 

Step 4. If Z*_=- Z p stop,_a minimal cut for the original network G(N. A) is at hand. 
(Z*. Z k ) - (Zp, Zp) is a minimal cut, with capacity c(Z k . Z k ) already calculated. 

If p - k < 3 (i.e., Z* and Z p differ by at most 3 nodes)^ then (Z*. Z k ) is a 
minimal cut for the original network if _c(Z k . Z k ) < c(Z p , Z p ) or (Z p . Z„) is a 
minimal cut for G if c(Z p , Z p ) < c(Z k , Z k ), or else both cuts are minimal cuts for 
G. 

If p — k > 3, then let (Z k , Z k ) and ( Z p , Z p ) divide the node_set of the original net¬ 
work into three subsets - Z k , N 2 = Z p - Z*, and A 3 - Z „, respectively. Con¬ 
struct the intermediate subnetwork G 2 (jVJ, /1 2 ), where N 2 U (s*, f), where s* 
and t* represent a super source and a super sink, respectively. Determine u s .j and 
u jt‘s j € N 2 as described before. Obtain the maximal flow for G 2 by using Karzanov's 
algorithm [12], At maximal flow apply Ford and Fulkerson’s [4] labeling algorithm 
once to obtain a minimal cut. The minimal cut obtained by this procedure is a 
minimal cut for the original network, G. 

Assuming Karzanov’s [12] algorithm requires 0(n 3 ) operations, we can determine the 
computational upper bound for the proposed algorithm. Let us assume that after one-node 
decomposition we have IZj - k , IZ„| « n - p, also let x - k + (n - p). Thus, the total 
number of additions and/or subtractions for Steps 0 through 3 can be written as 0(x/j 2 ). Simi¬ 
larly, the number of comparisons can be written as 0(xn). If k * 1 and p - n - l(x - 2), 
then the algorithm requires 0 (n 2 ) additions and 0 (n) comparisons and concludes that either 
(Z|. Z|) or (Z„_). Z,,.,) (whichever has the smallest capacity) is a minimal cut for G or else a 
minimal cut should be determined on the overall network by utilizing any efficient maximal 
flow algorithm available. 

If 2 < x < (n - 4) thenjhe algorithm_will produce an intermediate network G 2 which 
will lie between the cuts (Z k , Z k ) and (Z p , Z p ). Including the super source s* and the super 
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sink r*, Gi will contain n - x + 2 nodes. Finding the maximal flow in this network will give 
us a computational upper bound of Ol(/i - x + 2) 3 ] in Step 4, if Karzanov’s (121 algorithm is 
used. Assuming that the upper bound is reached in Karzanov’s algorithm, then the computa¬ 
tional upper bound of the proposed decomposition algorithm can be written as 0 ((r - x) 3 ). 

5. CONCLUSIONS 

Two alternative decomposition procedures have been presented. In the first procedure, 
one-node cuts were selected in a fixed predetermined order. If we naively assume that the 
values of one-node cuts are random and coming from the same distribution, then we can And 
out the expected numbers of runs, E(X), (increasing or decreasing blocks) from the formula 
E{X) - (2(/V - l) - l)/3 - (2N - 3)/3 and its variance V(X) - U6(/V - 1) - 

291/90- (16 N - 45)/90 given by Levene (141, where (N - 1) is the total number of observa¬ 
tions (one-node cut values). For (jV — 1) > 20, the distribution of X can be approximated by 
a normal distribution with mean E(X) and variance V(X) as given above 114J. 

Therefore, in a large-scale network, the probability of having X < 2 can be obtained from 
the statistic 

Z - “ ( ~ 2N + 9)/l(16 * " 45 >/ 10 l l/2 - 

For N - 22, P(X 2) - 4> (-6.317) - 1.0 x 10~ 9 . 

The probability obtained above indicates that the probability of having one or two blocks 
in (TV — 1) cuts is very small. In other words, the probability of hitting the worst case is almost 
zero (if the two blocks are an increasing and a decreasing one) as well as the probability of 
finding the minimal cut without further computation (if there is either one block, or two blocks 
starting with a decreasing one). However, if the arc capacities are identically, uniformly and 
independently distributed between two integers, a and b, and if the network is complete, then 

E(u lf ) - ntj - n - (b + a)f2 
and 

V(Ujj) - <rfj - a- 2 - (6 - a) 2 / 12 . 

Thus, the expected capacity of the cut (Z*, Z k ) can be written as 

E{c(Z k , Z k )} — k(n - k)n - k(n — k) (6 + a)/2 
and 

V[c{Z k . Z k )} - Ik in - k)] 2 ib - a) 2 /12, 

where kin - k ) is the number of arcs on the cut (Z k , Z k ) from the node_set Z k to the node 
set Z k . It is clear from the above formulas that the one-node cut (Z„/ 2 , Z„ /2 ) has the largest 
expected capacity and also the largest variance. 

We have simulated some random networks with arc capacities uniformly and indepen¬ 
dently distributed over the range (1, 50) and the number of increasing and/or decreasing 
sequences are given in Table 1 below. 

It is clear from the table that a sparse network can be decomposed into more subnetworks 
than a dense network. 
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TABLE 1 — Average Number of Increasing and/or 
Decreasing Blocks on Random Networks 


Number of 
Nodes 

Number of 
Replications 



Density UU 

n(n — 

1 ) 




.1 

.2 

.3 

.4 

.5 

.6 

.7 

.8 

.9 

1.0 

10 

50 

3.30 

3.92 

3.82 

3.36 

3726 

3.02 

2.80 

2.64 

2.44 

2.24 

20 

50 

6.5 

5.96 

5.48 

4.48 

4.16 

3.64 

3.76 

3.40 

2.64 

2.80 

30 

50 

10.02 

7.88 

6.56 

5.72 

4.96 

4.76 

4.24 

3.76 

3.40 

3.00 

40 

50 

11.52 

8.88 

7.20 

6.32 

5.72 

4.52 

3.32 

4.32 

3.60 

3.16 

50 

50 

13.78 

10.04 

8.82 

6.88 

6.08 

5.36 

4.36 

4.32 

3.88 

3.72 

75 

50 

18.08 

12.6 

9.68 

8.20 

7.12 

6.08 

5.48 

4.84 

4.40 

4.20 
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ABSTRACT 

In (his paper we consider dual angular and angular structured mixed integer 
programs which arise in some practical applications. For these problems we 
describe efficient methods for generating a desirable set of Benders' cuts with 
which one may initialize the partitioning scheme of Benders. Our research is 
motivated by the computational experience of McDaniel and Devine who have 
shown that the set of Benders' cuts which are binding at the optimum to the 
linear relaxation of the mixed integer program, play an important role in deter¬ 
mining an optimal mixed integer solution. As incidental results in our develop¬ 
ment, we provide some useful remarks regarding Benders' and Dantzig-Wolfe's 
decomposition procedures. The computational experience reported seems to 
support the expedients recommended in this paper. 


1. INTRODUCTION 

In this paper we consider some mixed integer programs with special structures. Our 
research is motivated by the empirical study of McDaniel and Devine [10] who discovered that 
when solving a mixed integer program using the decomposition scheme of Benders [2], the set 
of Benders' cuts or constraints in the master problem which are binding at the linear program¬ 
ming optimum are almost sufficient to obtain an optimal solution to the mixed integer program, 
in that only a few additional Benders' constraints are required. In fact, working with problems 
of different structures, McDaniel and Devine [10] found that if they were to solve the linear 
relaxation of the mixed integer program using the decomposition scheme of Benders [2], that 
is, solve the master program at each iteration as a linear program, then when they switched 
over to solving each master program as an integer programming problem after having obtained 
the linear programming optimum, very few (typically one) additional Benders' inequalities were 
required to verify optimality. 

In the light of this study, we examine two types of mixed integer programs, each with a 
special structure. For these programs, we describe a means of exploiting their special structures 
to obtain an initial set of Benders’ cuts with which one may initialize the usual partitioning 
scheme of Benders [2]. This initial set of cuts is desirable in view of McDaniel and Devine’s 
[10] observation, in that the linear relaxation of the master program with these cuts obtains an 
optimal or near optimal solution to the linear relaxation of the original mixed integer program. 
As incidental results in our analysis, we provide some interesting remarks regarding Benders’ 
[2] and Dantzig-Wolfe's [4] decomposition procedures. 


‘This material is based upon work supported by the National Science Foundation under Grant No. ECS-8103732. 
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2. MIXED INTEGER PROGRAMS WITH DUAL ANGULAR STRUCTURE 
In this section, we consider a mixed integer program of the form 

PI: minimize c{ x\ + c 2 x 2 + d'y 

subject to ^ 1*1 + D\y > b\ 

A 2 x 2 + Djy > b 2 
Gy > g 

x,. x 2 . y > 0 
y € n 

where n is the set of integer vectors and where a superscript t will throughout denote a matrix 
transpose operation. Problem PI is said to have a dual angular structure with the variables y 
being coupling variables (see (7] for example). For the sake of notational simplicity, we have 
considered only two diagonal blocks of x-variables, though of course, the development general¬ 
izes to any number of such blocks. 

Problem PI may arise, for example, within the context of discrete stochastic programs 
with recourse [131 where in addition, the "first stage decision" vector y is restricted to be integer 
valued (typically binary-valued). The proposed approach in this case becomes particularly 
attractive if the random right-hand side vector b can only be one of a reasonably few number of 
vectors b\. b 2 . b r according to some discrete probability distribution. 

Now, let LP1 denote the linear relaxation of Problem PI, that is. Problem PI with the 
restrictions y € II relaxed. Further, merely for the sake of simplicity, assume that each set of 
variables X| and x 2 has one component which is an artificial variable with a large positive 
coefficient in the objective function and a column of ones in its corresponding constraint- 
coefficient matrix. This assumption guarantees that a feasible solution to Problem LP1 exists 
for any fixed value of y € Y, where 

(1) Y - [y. Gy>g, y > 0} 

Using the partitioning scheme of Benders [21, Problem LP1 may be decomposed as 
min d'y 4- [min c'jXi + c' 2 x 2 : A jXi > b\ - D x y, xj > 0 

v*Y x ,.* 2 

A 2*2 ^ b 2 - Dzy, x 2 > 0) 


which may be rewritten (using duality) as 

(2) min d'y + [{max (b| - D l y)'ir ] : A\ir l < cj, W| > 0| 

t t 

+ [max (b 2 - Diy)'n 2 : A' 2 ir 2 < c 2 . n 2 > 0}} 
"2 

Finally, denoting 

(3) S| ~ [iT| :< C|. *1 > 0} 

(4) S 2 - {ir 2 : <4'jir j < c 2 , ir 2 > 0} 
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and observing that our assumptions imply that S) and S 2 are bounded. Problem LP1 may be 
written as Benders’ master problem 


BMP: 

minimize 

subject to 

d'y + zi + z 2 

(5) 


Zi ^ (b D|y)*»r^ for each j € /] 

(6) 

where 

z 2 ^ (b 2 - Djy )kr^ for each j € J 2 
y € Y 

(7) 


J i — { j : it 1 is an extreme point of Sj } 

(8) 


J 2 “ \jtt* is an extreme point of S 2 ) 


Let this problem yield an optimal vector y*. Then, the corresponding xf and xf which define 
an optimal solution (xf, xf, y*) to Problem LPI are obtained respectively from (2) as dual 
optimal vectors to Problems SP1 (y*) and SP2 (y*), where 

SPl(y): max {(b) - D i y)'v l ir,6Sil 

SP2(y): max ((6 2 ~ Djy)'v 2 ir 2 € S 2 }. 

However, not all the constraints in (5) and (6) may be necessary to define an optimal 

solution (zf, 2 $, y*) to Problem BMP. As shown by Benders [21, the following relaxation of 

BMP may be sufficient so long as, if (ij, z 2 ,y) solves BMP ( k , /), then i\ and z 2 are respec¬ 
tively the optimal objective values of Problems SP1 <J>) and SP2 CP): 

BMP(/c,/): minimize d'y + zj + z 2 

subject to 

(9) z { > (*| - Diyj'rr / for j - 1. k 

(10) z 2 > (ft 2 - Djy^l fory- 1./ 

y € Y. 

If this is not the case, say, zi < (6] - D 1 y) f tr* +I , where ir* +l solves SP1 (p), then an addi¬ 
tional Benders’ cut (or constraint) zj > (b\ - D\y)' ir* +l is appended to Problem BMP (k, /). 
In this manner, Benders’ [2] scheme iterates between the master problem BMP ( k, I ) and the 
subproblems SP1 (y) and SP2 (y) until for some k and /, the above termination criterion is 
satisfied. It is our purpose to show how one may construct such a problem BMP (k, I ) is a 
more efficient manner. Thereafter, as indicated by McDaniel and Devine (10], if one com¬ 
mences the application of Benders’ scheme on Problem PI with the master program 
BMPU, /), then a minimal amount of additional cuts of the type (9), (10) may be required to 
solve PI. 

The main property we use toward this end is the dual relationship between Benders' [21 
scheme and the Dantzig-Wolfe [4] decomposition method (see Lasdon [7], for example). 
Observe that the dual (LD1) to Problem LPI has a block diagonal (or angular) structure as 
shown below 
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LD1: maximize b\it x + b 2 ir 2 + gia 

subject to 

D/ir, + D' 2 ir 2 + Gfer < d 
ff, € S, 

w 2 € S 2 

a > 0. 

This problem is most amenable to the Dantzig-Wolfe decomposition method [4] which writes 
each it , € S ( as a convex combination rr, - £ X ijw j\ where £ A , y — 1, A u > 0, and simi- 

J*J JSJ\ 

larly expresses each ir 2 € S 2 . In this manner. Problem LD1 is written as the Dantzig-Wolfe 
master program. 

DWMP: maximize £ A (i{ irjO+ £ X 2j (b 2 irfy + a'g 

HJ X JiJ 2 

subject to 

1 A w (Din-/) + £ X 2y (£> 2 tt j) + Gb 4 rf 
y«y, >€y 2 

Z *ty“ 1 • Z * V ” 1 

y«A 

Ajy ^ 0, j € ; X 2 y ^ 0, / € a ^ 0. 

Again, not all the columns for X ly , j € J\ and X 2/ , j 6 i 2 may be necessary to solve LD1 
through DWMP. In particular, the following restriction of DWMP may be sufficient. 

k 1 

DWMP(Jt, /): maximize £ X ly {b\ir f) + £ X 2y (b' 2 wp + a'g 
y-1 y-1 

subject to 


(11) 

^X u (Diir/) + £X 2J (DJrrp + Gfe < tf 
/- 1 y- 1 

* 


(12) 

Z*.y» 1 

y-i 


(13) 

£x 2 , - 1; A„ > 0, j - 1. k,X 2J >0, j~\, ... 

y-i 

. /; a > 0 


so long as, if y, z u and z 2 are optimal dual variables associated with constraints (II), (12) and 
(13), respectively, then Z| and z 2 are respectively the optimal solution values of Problems 
SP1 (p) and SP2 (p). If this is not the case, say z t < ( bj - Djy)' nf +l , where nf +I solves 
SP1 (y ), then an additional column corresponding to nj 1 *' (with variable A|<*+|>) is appended 
to Problem DWMP ( k, I). In this manner, Dantzig-Wolfe’s [4] method iterates between the 
master problem DWMP (k, I) and the subproblems SP1 (y) and SP2 (y) until for some A; and 
/, the above termination criterion is satisfied. 
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Note the dual nature of the foregoing remarks (7], In fact, Problems DWMP and BMP 
are duals of each other and so are Problems DWMP (Ac, /) and BMP (Ac, /). Thus, once we 
have solved LD1 and terminated with some DWMP (Ac, /), one can easily write out the Prob¬ 
lem BMP (Ac, I) which solves LP1. The reason why it’s preferable to solve LD1 is of course 
the savings in effort whereby the Dantzig-Wolfe decomposition scheme maintains a basis of a 
fixed size as opposed to one of an ever increasing size when using Benders' scheme. This may 
be more pronounced if y is a vector of a few variables and Y has several constraints and less 
pronounced otherwise. 

We will consider next a mixed integer program with an angular structure. Although this 
structure does not lend itself to as straightforward an analysis as above, we are nonetheless able 
to make some pertinent observations and remarks which are supported by our computational 
experience. 

3. MIXED INTEGER PROGRAMS WITH ANGULAR STRUCTURE 

Consider a mixed integer program P2 with the following angular structure. 


P2: 

minimize 

c'x + d'y 


subject to 


(14) 


Cx + Dy > / 

(15) 


Ax b 



Gy > g 


x > 0. y > 0 and y € ft 

where ft is the set of integer vectors. Problem P2 may be used to model fixed charge produc¬ 
tion and location, and other related problems, where the vector y is constrained to be binary 
valued, denoting a yes/no type of decision, and the vector x represents production or allocation 
variables. As before, let LP2 be the linear relaxation of P2, that is, with the constraint y € ft 
relaxed, let Xbe as defined in Equation (1), and assume that the set 

(17) X - \x: Ax > b. x > 0) 

is nonempty. For simplicity, we assume that both X and Y are bounded in the following 
development. Also, we have considered just one block of constraints in (15), although it may 
be that*'- (x|, .... x,'), c'- (cj. c' r ), b'~ ( b\ , #), C - [C x .C r l, and 

A t 0 

*2 

A - 

0 • A, 

Again, we will assume that the vector x has an artificial variable as one of its components, with 
a large positive objective function coefficient, and a column of ones in the rows (14) and zeroes 
in the rows (15). This will then guarantee that for any fixed y € K, there exists a vector x such 
that (x, y) is feasible to LP2. 
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Now, one may partition Problem LP2 using Benders’ decomposition [2j scheme as fol¬ 
lows: 

min d'y + {min c'x : Cx > / - Dy, Ax > b, x > 0} 
y* r x 

= min d'y + (max (/ - Dy)' n + b'a: C'n + A’a < c, n ^ 0, a > 0). 

it.a 

Defining the sets 

(18) S = [(n, a): C'n + A 'a ^ c, n ^ 0. a ^ 0) 

(19) y» {/: (n J , a 7 ) are extreme points of 5} 

a relaxed Benders’ master problem with k Benders' cuts may be written as 

MP(K): minimize z 

subject to 

(20) z > d'y + (/ - Dy)hr p 4- bW for each p € K Q J 

y € Y 


where K *■ {1. k\. Then if (z, y) solves Problem MP(AD, one would solve the subprob¬ 

lem SP fp) or its dual SD (p), where 

SP0>): maximize [d'y + if - Dy)' n + bh: (n. a) € S) 

and its dual is 

SDOO: minimize [d'y + c'x: Cx > / - Dy. Ax ^ 6. x > 0). 

If it turns out that z - d'y + c'x where x solves SD (p), then (x, y) solves LP2. Otherwise, 
z < d'y + c'x, and a Benders’ cut z ^ d'y + (/— Dy)'ir* +1 + b'a k+{ is appended to rows 

(20), where (w* 4-1 , <** +1 ) solves SP(y). The process now continues in this manner. 

Note that every time a new row is added to (20), a solution to Problem SD(y) is called 
for, a problem with an angular structure. However, Problem LP2 itself has just one more diag¬ 
onal block than Problem SD (y). Therefore, the application of Dantzig-Wolfe’s [4] decomposi¬ 
tion to the former problem has a basis of dimension one more than that applied to the latter 
problem. Moreover, while using Dantzig-Wolfe’s decomposition, we would be able to exploit 
any special structures which the set Y may have. This advantage may be lost in the solution of 
problems MP(ZO for [AT | > 2. Hence, we propose to do the following. We will solve Problem 
LP2 using Dantzig-Wolfe’s decomposition method (4], From this solution process, we will 
show how one may obtain a desirable set of cuts (20) so as to avoid expending effort in gen¬ 
erating cuts which may not be binding at the linear programming optimum. From this point 
onwards, one may continue with the solution of Problem LP2 using Benders' decomposition 
and then switch over to solve Problem P2 as advocated by McDaniel and Devine (101. In con¬ 
clusion, we will also make a remark which may be a further expedient in the latter process. 
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First of ail, let us present a pertinent observation afforded by Baias and Bergthaller [1]. 
Note that if Or*, y*) solves LP2, then the set of Benders' cuts which are binding at optimality 
are indexed by the following subset of the set J of Equation (19) 

(21) 7 - [j € J : (n J , a J ) is optimal to Problem SPCy*)). 

However, not all the cuts indexed by j € 7 may be necessary to determine y* in the Benders' 
master problem at optimality. In particular, if y* has 8 positive components, then SD(y*) has 8 
degenerate basjc variables at optimality, and so SP(y*) has up to 2 s alternative optimal solu¬ 
tions. Thus, |7| < 2* with the equality often holding [1]. But the dimension of Benders' mas¬ 
ter program is n + 1, where y has n components. Further, we know that n - p of the hyper¬ 
planes y, > 0 / € {1.n) hold as an equality at y*. Thus, at most, some 

(n + 1) - (n - p) = p + 1 Benders' cuts indexed by 7may be needed to define the linear pro¬ 
gramming optimum y*. We now proceed to demonstrate how one may generate a desirable 
subset of cuts indexed by J. Our discussion will motivate the choice of this subset. In addi¬ 
tion, we will show how some more Benders' cuts which are binding at optimality may be 
recovered during the application of Dantzing-Wolfe’s decomposition procedure to Problem 
LP2, without any extra effort. 

Toward this end, suppose that we have solved Problem LP2 using Dantzig-Wolfe’s 
decomposition method. Let the Dantzig-Wolfe master program at optimality be 


DWMP2: 

minimize £ X p ( c%) + j, y q ( d'y q ) 

(22) 

P- 1 Q -1 

X X, ( Cx„) + £ y q ( Dy v ) > / 


p- • </-1 

(23) 

X K P ” I- x, > 0 for p - 1. . 

p- 1 

(24) 

X Yq “ 1 • Yq ^ 0 for q - 1, . 
<?" 1 


Suppose (X *. y *) solves DWMP2 with X * > 0 p - 1. k, y* > 0 q - 1./. Then 

Oc*. y*) solves LP2 where x* - £ X f(x p ), y* - £ y* O',). Here, x„c vert X (Equation 

P— I <7-1 

(17)) and vert Y (Equation (ID). Further, if it *, 9 * and <t> * are optimal dual variables 
associated with the constraints (22), (23) and (24) respectively, then 0 * and 4 * are respectively 
the optimal solution values of Problems SPx(ir *) and SPy(rr *), where 

SPx (it ): minimize |(c - C*ir)'x : x € AT) 

SPy(ir): minimize {(o' - D*ir)'y : y € Y}. 

Lemma l below recovers a dual optimal solution to Problem LP2 from the Dantzig-Wolfe mas¬ 
ter and subproblems at optimality. This result may be viewed as an extended special case of 
the more general result due to Magnanti, Shapiro, and Wagner [8). 
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LEMMA 1: Let Problems DWMP2, SPx(ir) and SPy(ir) and the vector it * be as defined 
above. Further, let a * and 0 * be optimal dual variables in Problems SPx(rr *) and SP>(w •), 
respectively. Then, (ir *, a *, 0 *) is an optimal dual solution to Problem LP2. 

PROOF. From the duals to Problems SPx (n *) and SPy(w*), it follows that 
(it *, a *. 0 *) is dual feasible to LP2. Further, with x*, y \ 9 * and <t> * as defined above, the 
Dantzig-Wolfe termination criterion implies that 9 * - b'a * and <b * " gfi *■ Thus, from 
DWMP2, c'x* + d'y * fn * + b'a * + gfi *. This completes the proof. 

Noting that (w *. a *) € 5, let us construct the following Benders’ master problem with a 
single cut constraint. 

MP(*): minimize z 

(25) subject to z ^ d'y + (f - Dy)*n * + b\a* 

y e K 


Now, consider the following result. In view of Lemma 1, this result is closely associated with 
the concept of "strongest surrogate constraints," a name given to constraints of type (25). (See 
Geoffrion 15], Glover [6] and Rardin and Unger [11}). 

LEMMA 2: Let (x*, .v*) and (it*, a*. 0*) be optimal primal and dual solutions respec¬ 
tively to Problem LP2. Then y*is optimal to MP(*) with z* = c'x* + d'y*. 

PROOF: A Lagrangian dual to Problem LP2 is 

LD: maximize (hiir. a) : v > 0, a > 0} 

where h(n. a) — min [d'y + (/ — Dy)Sr + b'a] + min ((c - Cn — A'a)'x]. 


Since there is no duality gap between LP2 and LD, we know that (ir • a *) solves LD with 
objective value /»(«•*. a*) equal to c'x* + d'y*. Further, (x*. y*) evaluates h(n*. a*). But 
by complementary slackness in LP2, the second problem in h(n *, a *) has a value of zero at 
x — x*. Thus, the first problem in h(ir *. a *), which is Problem MP(*), has an optimal solu¬ 
tion y* and an optimal value of c'x* 4- d'y*. This completes the proof. 

Let us digress momentarily to discuss a certain issue related to the strongest surrogate 
constraint (25). Often, such a constraint is prescribed as a device for somehow fathoming par¬ 
tial solutions, or selecting branching variables in a branch and bound or implicit enumeration 
context, implemented within the framework of an algorithm which solves Problem P2 using 
Benders’ decomposition. Lemma 3 below addresses the case when such a strategy may not be 
very meaningful, that is, when (25) degenerates to simply z ^ z*. 

LEMMA3: Let (x*, ^*) and (ir *, a *. 0 *), respectively, be optimal primal and dual solu¬ 
tions (of value z*) for Problem LP2. Then, if y* € int F, the strongest surrogate constraint 
(25) degenerates to z ^ z*. 
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PROOF: If y* € int K, then j3 * = 0 and y* > 0. Thus, z* - fir * + fa * and the dual 
constraints Dhr + Gj3 ^ d are binding at («■. /3) — (it *, 0*). In other words, D'rr * — rf. 

Noting that (25) is z > C/% * + 6k *) + (rf - Dht *)'>>, the proof is complete. 

COROLLARY: In case (25) does not degenerate to z > z*, then y* and hence each 

/ 

.v, € vert K, where v* - ^ y J belong to the face of Y defined by its intersection with the 

q-[ 

hyperplane (d - Dhr *)'y = gfi *. 

The above result holds trivially from Lemma 2 and Equation (25). Its significance is the 
subject of our next discussion. 

Now, Lemma 2 seems to indicate that a very desirable starting point at which to com¬ 
mence the solution of Problem LP2 by Benders' decomposition is with Problem MP(*). In 

addition to this single cut, one may also choose to generate the following desirable set of 
independent cuts which will be binding at the linear programming optimum, i.e., which are 
indexed by the set J. 

The reader may note from the corollary to Lemma 3 that each y q , q =* 1. I is an 

alternative optimal solution to MP(*). Therefore, it is intuitively appealing to generate / 

Benders' cuts in addition, one for each y q through Problem SPfy,,) or SD(y q ) for q **= I./. 

However, these cuts may not be binding at the linear programming optimum, that is, they may 
not be indexed by the set J. To overcome this, consider the following subproblem which 
differs from SP (y q ) by the additional constraint (26) 

SP '(y q ): maximize d'y q + (/- Dy q )*x + b<a 

subject to ( 7 r, a) € S 

(/ - Dy*)' ir + b'a ^ z* - d'y* — c'x*. 

Lemma 4 below characterizes a Benders’ cut obtained through Problem SP'O^). Thereafter, we 
show how to solve SP'(>> ? ) while still exploiting the angular structure which SDCv,) possesses. 

LEMMA 4: Consider any y q , q € (I./! and suppose that (ir q . a") solves Problem 

SP'Ov). Consider the Benders’ cut 

(27) z > d'y + <J - DyY it" + fa". 

Then the cut (27) satisfies q € J. Moreover, when this cut is appended to Problem MP(*), it 
deletes y q provided there does not exist an x q € X such that (x„. y q ) solves LP2. 

PROOF: Since (z*. y*) is an optimal solution to MP(7) where J is defined in Equation 
(19), it follows that 

(28) z* > d'y * + (/ - Dy•)* + fa 

for any (it. a) € S. Noting Equations (26) and (28), we may assert that the cut (27) is 
indexed by q € / Now, y q is an alternative optimal solution to MP(*). Thus, if (x q . y q ) does 
not represent an optimal solution to LP2 for some x q 6 4f, then there must exist some u € J 
such that z* < d'y q + (/ - Dy q Y ir u + bb r". But (ir“, a") is feasible to SP'Cv,). Hence, we 
must have z* < d'y q + (/ - Dy q )'n‘> + bit". This completes the proof. 
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We will now proceed to show how one may solve SP'Cy,) with no more effort than that 
required to solve SPCfy) or SD(y,). Observe that for a fixed y - y* in LP2, we obtain 
2 * - d'y* — c'x* as the maximum value of (/ — Dy*) i n + b'a over (n, a) € S. Noting Equa¬ 
tion (26), Problem SP'(y,) may be viewed as a preemptive priority multiobjective program over 
the set S, with an objective (f - Dy*)'n + bit having first priority and an objective 
(/— Dy q )'ir + b'a having a lower priority. According to Sherali and Soyster [12], there exists 
a positive scalar £ * such that for any £ ^ £ *, an optima) solution to the following problem is 
also optimal to S?'(y q ). 

d'y q + maximum {[(1 4- £)/ - Dy q - £ Dy*]'n + (1 +• £)6'a : (ir, a) 6 S). 

The dual to this problem is the following angular structured linear program. 

SD'Ov £): d'y q + minimum (c'x: Cx > (1 + £)/- Dy q - £ Dy* 

Ax > (1 +t)b. x > 0|. 

Note that finding a £ large enough (£ > £ *) should be no problem since it is easy to show [12] 
that £ * in our case is simply the negative of the optimal dual variable associated with constraint 
(26). Moreover, we know that we have chosen £ large enough after solving SD’(y,, £) by veri¬ 
fying that (26) is binding at the optimal solution (ir q , a"). 

Summarizing, one may first solve LP2 using Dantzig-Wolfe's decomposition and retrieve 
from that (through Lemma 1) the strongest surrogate constraint (25). Then the master pro¬ 
gram MP(*) with this single cut constraint determines the optimal value of LP2. However, 

since any point in the convex hull of {yi. y t ) is also optimal to MP(*), one may determine 

for each q - 1./ a Benders’ cut which will be binding at optimality and will also delete y q 

by solving SD'(y v , £) for a large enough value of £ as indicated above. In concluding this sec¬ 
tion, we provide below two remarks. The first of these pertains to an alternative strategy for 
generating an initial set of Benders’ cuts and the second remark further addresses the issue of 
strongest surrogate constraints. 

REMARK 1: This remark is an alternative to generating an initial set of Benders' cuts via 
Problems SD'Cv,, £). Note that without any extra effort, during the solution of Problem LP2 
by Dantzig-Wolfe’s decomposition method, certain Benders’ cuts are automatically available. 
As the following discussion motivates, these cuts are quite likely to be binding at optimality. 
Suppose we execute the Dantzig-Wolfe procedure as follows. At any stage, having generated 

extreme points >»[.y v of Y in the master program, we continue to iterate between the 

master program and the subproblem SPx(n-) only until no more extreme points of X need to be 
generated. At this point in time, the current solution is clearly optimal to the problem 


DW(v): 

minimize 

c'x + j y q ( d'y q ) 

(29) 

subject to 

Cx + £ y q ( Dy q ) > f 

(30) 


Ax > b 

(31) 


i ><, - 1 

qm\ 

x ^ 0, y q > 0 for q - 
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Currently, suppose that it \ 0 v and 4 > v are optimal dual variables associated with the constraints 
of the type (22), (23) and (24), respectively, and thata v is an optimal dual solution to the sub- 
problem SPx(jt v ). Then following Lemma 1, ir v , a v and <t > v may be shown to be optimal dual 
variables associated with constraints (29), (30) and (31), respectively. But this means that if 

(y. x) solves DW(\), then for a fixed y - J, y q y q> x and (w v , a v ) are, respectively, optimal 
_ _ q- i 

solutions to SD(y) and SPiy). Therefore, a Benders' cut 

(32) z > d'y + (/- DyY ir v + b'a* 
is readily available. 

Hence, if we execute Dantzig-Wolfe’s procedure by generating as many extreme points of 
X as required before we generate an additional extreme point of Y, then for each optimal com¬ 
bination of extreme points of Y which have been generated at any stage, we have a Benders' 
cut (32) available from the current dual variables. It is intuitive that these cuts are desirable. 
Of course, at termination, (32) is precisely (25). Also, one may finally choose to keep those 
cuts (32) which turn out to be binding at the linear programming optimum. We provide some 
computational experience in Section 5 regarding this remark. 

REMARK 2: This note is related to the strongest surrogate constraint (25) and to our 
comments associated with Lemma 3. Observe that by also accomodating the constraints 
Gy ^ g into the objective function of Problem LD in the proof of Lemma 2, it is easy to show 
that the following cut may be used in lieu of (25) in Problem MP(*). 

z ^ d'y + (./ - DyY it * 4- hint * + (g - Gy)J3 * 
or 

z ^ (fir* + b'a* + g'l3*) + (d - D'rr* - G'/3*)'y. 

This is again the strongest surrogate constraint of Rardin and Unger [11]. But fir *_+ b'a* + 
g'(3* — z*, the optimal objective value of Problem LP2, and (d - D'n* - G'fi*) - d > 0, the 
reduced cost coefficient vector of the ^variables at the linear programming optimum. Thus, 
the above cut is 

(33) z > z* + d'y. 

Observe that if y* has all positive components, then d - 0, whence (33) is simply z ^ z*. 
Also, it is easy to see that if (25) degenerates to z ^ z*, then so does (33), though not neces¬ 
sarily vice versa. For example, g)3*^0 is sufficient for (25) not to generate to z > z*, 
though not for (33). Moreover, since our master program treats the constraint set y € Y expli¬ 
citly, we find (25) more appropriate. 

4. ILLUSTRATIVE EXAMPLE 

Consider the following problem adapted from Lasdon [7], 

P2: minimize|-2x! - x 2 — y\- yi- ~ 2xi - x 2 - yi - 2y 2 > -40. x € X. y € Y 

and y integer! 

X - {x: -x, -10, -xj S* -10. -x, -x 2 > -15. x,. x 2 ^ 0| 

Y - !y: ~y\ - 3y 2 S* -30. -2y, -y 2 > -20. y,. y 2 > 0). 
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Using Dantzig-Wolfe’s decomposition to solve LP2, we first generate a vector y by solving the 
problem: minl-^ - y 2 - y e H- This yields y - (6, 8). Continuing with the usual procedure 
but generating only vertices of X as needed, that is, solving DW(v - 1), yields x - (4, 10) as a 
convex combination of the vertices (0, 10) and (5, 10) of X. For this solution, ir v - 1, 
b'a* - 0 and hence the cut (32) is 

(34) z > y 2 - 40. 

The next vertex of Y generated through SPy(ir v ) is y — (10, 0). Now, v - 2 and the 
solution to DW(v- 2) through the continued generation of extreme points of X yields 

x - (10. 5), a single vertex of X , and y — •—■(■ 6,8) + ~ (10, 0) ” -y-j. At this 

stage, ir v — y, b'a y — —■— and hence the cut (32) is 

(35) z > - j y,-j y 2 ~ 30. 

However, no more vertices of Y need to be generated and so x* = (10. 5) and y* — 
solves LP2 with optimal value z* - - -y^-. Thus, (35) is actually the strongest surrogate con¬ 
straint (25). Note that according to Remark 2, that cut (33) is simply z > - -y-. Further, 
we obtain 

MP(*): min {z: z > - y y\ ~ y y 2 - 30. y 6 F). 

This problem has (6, 8) and (10, 0) as alternative optimal extreme point solutions with a value 
of z* - — y-. Note that the incidental cut (34) automatically obtained during the solution of 

LP2 is binding at the linear programming optimum. Indeed, this is also the cut which would be 
obtained through SD'(.y,. 0) with y Q = (6, 8). The reader may easily verify that the problem 

SD'O’,. f), for y t - (10. 0) and for any £ ^ 0. yields an optimal dual solution {it", a") with 

jt«- 0 and bit" - 25. Hence, corresponding to y v - (10, 0), the Benders’ cut which is binding 
at optimality is 

(36) z >-y t - y 2 ~ 25. 

It turns out here that the cuts (34) and (36) imply the strongest surrogate constraint (35). This 

1 2 

may be verified by using the surrogate multipliers — and — on (34) and (36) respectively. 

Thus, the master program MP (AO with the cuts (34) and (36) yields in this case an optimal 
solution to LP2 using the fewest number of Benders' cuts. In general one would have contin¬ 
ued solving LP2 with the Benders’ cuts (34), (35) and (36). 

Now, continuing with the solution to Problem P2, we obtain an optimal solution to 
MP(Af) with y restricted to be integer valued as y - (8, 4) with z - -36. Solving the subprob¬ 
lem SD(y), we obtain an optimal solution x — (10, 4) also with value - 36. Thus, (*, >>) 
solves P2 and we terminate without having to generate any further Benders’ cuts. Indeed, such 
a propensity is borne out by the computational experience of McDaniel and Devine [101. 


25 H) 
3 ’ 3 
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5. COMPUTATIONAL EXPERIENCE 

In this section, we will attempt to support the ideas recommended in this paper through 
some computational experience. Toward this end, recall the principal thrust of this paper. 
McDaniel and Devine (10] have amply demonstrated that in solving a mixed integer program 
via Benders’ 12] procedure, an attractive strategy is to solve its linear relaxation first via 
Benders’ partitioning method, and use the resulting cuts as an initial set of cuts for the former 
problem. Hence, we have attempted in this paper to demonstrate how one may solve the linear 
relaxation via Benders’ decomposition method more efficiently, for some specially structured 
mixed integer programs. 

The discussion in Section 2 dearly demonstrates the value of the suggestions for dual 
angular structured problems, particularly when there are few ^-variables and several more con¬ 
straints in Y. On the other hand, the suggestions in Section 3 need computational support. 
More specifically, one needs to demonstrate how well the proposed initial set of Benders’ cuts 
represents the linear programming solution, as wetl as the savings in effort which accrues from 
their use. Furthermore, the comments in Remark 1 need to be tested. 

Hence, we performed the following brief computational experiment on an IBM 370 Model 
158 computer with coding in Fortran. We generated linear programs of the form to minimize 
c'x + d'y, subject to coupling constraints Cx + Dy ^ /, x > 0, v > 0 and block constraints 
Gy > g, y > 0, with c and d generated uniformly on [-11,-1], C and D generated uniformly 
on [-50, 0], /generated uniformly on [—251, —1], # generated uniformly on [—6, —1] and with 
G having components 0 or -1 with an expected density of 0.5. All generated coefficients were 
rounded off to the nearest integer. 

Table 1 summarizes the results. The "Usual Benders’ Method” is being performed by 
treating the set Y - {y: Gy ^ g, y ^ 0] as the set of complicating variables. The "Section -3- 
Methotf generates an initial set of cuts via problems SD'( •) and then continues as usual, 
whereas the "Remark-l-Method" employs the recommendations embodied in Remark 1 of Sec¬ 
tion 3 to generate an initial set of cuts. 

Essentially, both the methods of Section 3 perform better than the usual Benders' 
approach, with that of Remark 1 appearing somewhat superior. However, as in Problems 1 and 
2, the Section-3-Method seemed better than the Remark-l-Method in some other larger sized 
problems we attempted in which NB >> NC. The reason being that the Remark-l-Method 
generates cuts from SP(y) where y is a combination of some extreme points of F, and when¬ 
ever the Dantzig-Wolfe algorithm generates and drops off several ^-vectors while solving LP2, 
the Remark-l-Method cuts are not as effective. 

In conclusion, we reiterate that the savings in effort as reflected through columns / would 
have been more enhanced if Y had a special structure, since this could have been exploited in 
solving problems SPy(-) though not necessarily in solving MP(A) for \K | > 2. 

6. SOME CONCLUDING REMARKS 

in conclusion, we present two further expeditious remarks. The first of these pertains to 
iteratively updating solutions to the subproblems SD(y) rather than resolving each subproblem 
independently. Essentially, this remark in general contributes towards performing sensitivity 
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TABLE 1 — Computational Results 


Problem 

Number 

m 

n 

NC 

NB 

Usual-Benders 

Method 

_ 

Section-3-Method 

Remark-1-Method 

a 

b 

c 

d 

e 

t 

c 

d 

e 

f 

1 


o 

30 

40 

12 

3 


~T" 

~T~ 

0.427 

~7” 

~T~ 

1“ 

0.445 

2 

13 

E3 

10 

40 

5 

3 

4 

4 

4 

0.760 

4 

5 

4 

0.900 

3 

40 

40 

30 

40 

6 

3 

4 

4 

4 

0.850 

4 

4 

4 



30 

30 

10 

40 

6 

3 

4 

5 

5 

0.973 

5 

5 

4 

0.757 


30 

30 

30 

40 

9 

3 

4 

5 

5 

0.648 

4 

4 

4 

K » ylM 


20 

20 

10 

40 

4 

3 

4 

4 

4 

0.768 

3 

3 

3 

EE II 


50 

50 

40 

30 

8 

2 

3 

3 

3 

0.626 

4 

4 

2 

0.431 

8 

50 

50 

40 

10 

8 

3 

4 

4 

4 

0.835 

5 

5 

4 

ran 

■j 

40 

40 

40 

30 

7 

2 

3 

3 

3 

0.537 

2 

2 

2 

0.326 


30 

30 

40 

10 

5 

3 

4 

5 

5 

1.379 

3 

3 

3 

0.621 

mm 

30 

30 

40 

30 

5 

3 

4 

5 

5 

1.270 

3 

3 

3 

0.622 

KB 

20 

20 

40 

10 

10 

3 

n 

4 

4 

0.625 

3 

3 

3 

0.286 


50 

50 

40 

40 

8 

2 

El 

3 

3 

0.604 

4 

4 

2 

0.432 


40 

40 

40 

40 

7 

2 

D 

3 

3 

0.477 

2 

2 

2 



30 

30 

40 

40 

5 

3 

n 

5 

5 

1.190 

3 

3 

3 

KB 


20 

20 

40 

40 

10 

3 

n 

4 

4 

0.494 

3 

3 

3 

Isa 


(i) m - # of x variables; n ” # of y variables; NC — # of coupling constraints; NB - # of block constraints (in Y). 


(ii> a = total number of Benders' cuts required; 6 s # of cuts in V which were binding at optimality. 

(iii) <■ s # of initial cuts (including strongest surrogate constraint); d = total number of cuts required; e s # of cuts in 
-rf- which were binding at optimality. A = ratio of total solution time to the total solution time for the Usual- 
Benders-Method. 


analysis in the context of Dantzig-Wolfe's decomposition procedure. However, the question 
regarding its computational efficiency is open to further research. The second remark briefly 
addresses the issue of effectiveness of Benders' partitioning scheme. 

REMARK A: This remark concerns the iterative solutions of the subproblems SD(y), 
both, while continuing to solve LP2 as well as while subsequently solving P2 using Benders' 
decomposition. Suppose that we have solved SD(y) for some y using Dantzig-Wolfe's decom¬ 
position. (Here, Ax > b may be one of several angular blocks of constraints). Letir and# be 
optimal dual multipliers for the constraints Or > / - Dy and the convexity constraint for the 
extreme points of X , respectively. Then, on replacing y by some y for the subsequent solution 
of Problem SD(y)> we have a dual feasible but primal infeasible Dantzig-Wolfe tableau. Let 
the Ah row in this tableau have a negative right-hand-side value and denote the first m columns 
of the Ah row of the current basis inverse by B„ where 0 has m rows. Further, let the element 
of the basis inverse in the Ah row and the same column as the row of the convexity constraint 
for the x-variables be <l> Then, it follows that the reduced cost coefficient (c - CW)'x - 9 is 
nonnegative for each x € vert X. Now, in order to perform a dual simplex iteration in row /, 
we need to generate a column corresponding to a vertex x of X such that its updated value 
BiCx_+>l/ / in row i is negative and yields the smallest value for the ratio |(c- CWY 
x —0)1 - j B,Cx +0,) among all such columns. To accomplish this, consider the following 
linear fractional program. 
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LFP: minimize | - C ■ ^ X - - : Ax > b, x > o|. 

( -\BiCx +$,] j 

Note that if one solves this problem using Chames and Cooper's [3] method by letting 
u - (1/ - {B,Cx +!>;!) > 0 and v - ux, then at optimality, we will have u > 0 since SD(y) 
is feasible by our original assumption. This transformation converts LFP to the linear program 

minimize {(c — Cw)' v — 6u: A\ — bu > 0, B,Cv + t|i,u — —1. u > 0, v ^ 0). 

Thus, instead of solving each subproblem from scratch using Dantzig-Wolfe’s decomposition, 
one may update the solutions to subsequent subproblems through dual simplex iterations as 
indicated above. Note that this is a general technique which one may adopt to perform a sensi¬ 
tivity analysis on right-hand-side perturbations within the context of Dantzig-Wolfe’s decompo¬ 
sition method. 

REMARK B: It has been conjectured that the "nearer'’ the solution to the linear relaxa¬ 
tion of a mixed integer program is to a solution of the orginal mixed integer program, the fewer 
the number of Benders' cuts that will be required to solve the mixed integer program. The dis¬ 
cussion is this paper tends to support this statement in the following manner. 

Suppose that it turns out that LP2 has an optimal solution (x*. >>*) satisfying 
y* e Y n fi. Let 2 * - c'x* + d'y* and let (ir *, a * 0 •) be a dual optimal solution to Prob¬ 
lem LP2. Then, the strongest surrogate constraint (25) is also the Benders' cut which would be 
generated through SP (>»*). Hence, using Lemma 2, it follows that the Benders’ master problem 

minimize z 

subject to z > d'y + (/ - DyYit * + b\x • 

y e y n a 

with the single cut (25) has (z*. >*) as an optimal solution. Thus, it is possible for the pro¬ 
cedure to terminate after generating a single Benders’ cut. In any case, if one is applying the 
usual Benders' scheme to solve Problem P2, then the procedure will terminate at the particular 
iteration at which the current master problem yields y* (of value z*) as an optimal solution. 

ACKNOWLEDGMENT 

Thanks are due to a referee for helping us highlight the contributions of this paper. We 
would also like to transmit the referee's comment that the idea embodied in Remark B above is 
treated in greater detail by Magnanti and Wong (9). 

REFERENCES 

(1) Balas, E. and C. Bergthaller, "Benders' Method Revisited," Management Sciences Research 

Report No. 401, Carnegie-Mellon University (1977). 

[2] Benders, J.F., "Partitioning Procedures for Solving Mixed-Variables Programming Prob¬ 

lems," Numerische Mathematik. 4, 238-252 (1962). 

(31 Chames, A. and W.W. Cooper, "Programming with Linear Fractionals," Naval Research 
Logistics Quarterly, 9, 181-196 (1962). 

(41 Dantzig, G.B. and P. Wolfe, "The Decomposition Algorithm for Linear Programming," 
Econometrica, 9, 4 (1961). 


VOL 28. NO 3. SEPTEMBER 1981 


NAVAL RESEARCH LOGISTICS QUARTERLY 









462 


H. O SHERALI 


[5] Geoffrion, A.M., "An Improved Implicit Enumeration Approach for Integer Program¬ 
ming," Operations Research, 17, 437-4S4 (1969). 

{6] Glover, F., "A Multiphase-Dual Algorithm for the Zero-One Integer Programming Prob¬ 
lem," Operations Research, 13, 6, 879-919 (1965). 

[7] Lasdon, L.S., Optimization Theory for Large Systems, Macmillan Series in Operations 

Research (The MacMillan Company, New York, 1970). 

[8] Magnanti, T.L., J.F. Shapiro and M.H. Wagner, "Generalized Linear Programming Solves 

the Dual," Management Science, 22, II, 1195-1203 (1976). 

[9] Magnanti, T.L. and R.T. Wong, "Accelerating Benders Decomposition: Algorithmic 

Enhancements and Model Selection Criteria," M.I.T. Operations Research Center 
Working Paper, OR 093-79, October 1979; Operations Research (to appear) 

{10} McDaniel, D. and M. Devine, "A Modified Benders' Partitioning Algorithm for Mixed 
Integer Programming," Management Science, 24, 3 (1977). 

[Ill Rardin, R.L. and V.E. Unger, "Surrogate Constraints and the Strength of Bounds Derived 
from 0-1 Benders’ Partitioning Procedures," Operations Research, 24, 6 (1976). 

[12] Sherali, H.D. and A.L. Soyster, "Preemptive and Non-preemptive Multi-objective Pro¬ 

gramming: Relationships, Characterizations and Counter-Examples," Blacksburg, VA, 
Virginia Polytechnic Institute and State University, Working Paper (1980). 

[13] Walkup, D.W. and R. Wets, "Stochastic Programs with Recourse," SIAM Journal of 

Applied Mathematics, IS, 1299-1314 (1967). 


naval research logistics quarterly 


VOL 28, NO. 3. SEPTEMBER 1981 








OPTIMAL CONTROL OF PRICE THROUGH 
RESTRICTED PRODUCTION 


Steven A. Lippman 

Graduate School of Management 
University of California at Los Angeles 
Los Angeles, California 

Wayne L. Winston 

Graduate School of Business 
Indiana University 
Bloomington. Indiana 

ABSTRACT 

Consider a regulated monopolist whose current profits would be maximized 
if they could charge a price p, where p exceeds the current market price. By 
reducing production below current consumer demand the monopolist can create 
an illusion of a shortage and induce the regulator to allow a price increase. 

Conditions are given for which the production rate that maximizes the 
monopolist's expected discounted profits over an infinite horizon will have the 
property that the amount of unsatisfied consumer demand will be a nonincreas¬ 
ing function of surrent market price. 

1. INTRODUCTION 

Consider a monopolist Facing a demand curve Dip). If the cost of producing x units is cx 
then the firm would like to charge the profit maximizing price p, maximizing the revenue 
ip - r) Dip). Due to regulation the monopolist cannot immediately attain a price of p. How¬ 
ever, by producing less than Dip), the demand associated with the price p, the monopolist can 
create the illusion of a shortage; this shortage induces the regulatory agency to increase the 
price. Although the economic grounds for the regulator to base his pricing decisions on the 
observed level of excess market demand may be scant, regulatory agencies in several industries 
behave in this fashion. For example, the degree of market disequilibrium in the natural gas 
industry is a crucial factor in the determination of the regulated price. Owen and Brautigam 
note [9], that "Producers are sometimes alleged to ‘hold back' or delay production until regula¬ 
tors raise prices in the future." 

Assuming the current price is p with p < p and p > c, the monopolist must trade off the 
profits lost by not satisfying a portion of current demand against the future profits that can be 
gained by holding back production in order to induce a price increase. Our approach utilizes 
the concept of continuous time and semiMarkov decision processes. (Markov decision models 
in which the firm controls price and not production have been analyzed elsewhere (see [3], (4], 
and (8).) At each point in time the state of the process is the (regulated) price at which the 
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monopolist's production can be sold; the monopolist's action is the rate at which the product is 
produced or, equivalently, the amount of consumer demand the firm chooses to leave 
unsatisfied. The regulator's willingness to increase price depends on the current price as well as 
the consumer demand that the firm leaves unsatisfied. Thus, it is natural to attempt to deter¬ 
mine whether the optimal production strategy of the regulated monopolist increases in the regu¬ 
lated price p and, more importantly, whether the excess or unsatisfied consumer demand associ¬ 
ated with its production strategy decreases as the price increases. Our goal is to derive condi¬ 
tions which ensure that the amount of unsatisfied consumer demand is a nonincreasing function 
of the price. 

In Section 2 we describe the two models analyzed in this paper. Sections 3 and 4 are 
devoted to a characterization of the manner in which the firm’s optimal production strategy 
depends on parameters such as the current price, the discount factor, and the length of the 
planning horizon. Finally, Section 5 suggests an extension of our model. 

2. MODEL DESCRIPTION 

In both models to be considered the monopolist observes the price p at which its product 
can be sold. This price is set by the regulatory agency. When the price is p, demand for the 
product occurs at an instantaneous rate Dip). While the price is p and the monopolist’s output 
rate is y < Dip), the monopolist earns profit at the rate ip - c)y, where c is the true constant 
marginal cost of production. 

Our two models differ in the manner in which the monopolist’s production decision 
affects the evolution of the product’s price. In model 1 we assume that at any instant during 
which the price is p and the amount of unsatisfied consumer demand is x, the price increases to 
p + 1 in a time that is governed by a Poisson process with rate Kip. x). Naturally, that amount 
x of unsatisfied consumer demand satisfies x - Dip) - y. In model 2 the price increases from 
p to p + L * — 1. 2. 3.... with probability fik) after a time that is governed by a Poisson 
process with rates A (x). For both models we assume profits are continuously discounted at a 
rate a and the monopolist wishes to maximize the expected discounted profit earned over an 
infinite horizon. 

The firm is not allowed to select the price but rather is restrained by the regulatory 
agency. Determining the firm's marginal cost curve is no easy task for the regulatory agency 
and in this case the agency will make its inferences based upon the firm’s response to the 
agency’s actions.* 

The agency presumes that the firm has an increasing marginal cost curve as depicted in 
Figure 1. With the price given and p < P K the firm will set its production so as to equate mar¬ 
ginal cost and price. Moreover, it is clear from Figure 1 that if the firm’s actual marginal cost 
curve were as shown, then excess demand (D(p) — y) decreases with price while production 
increases. From the agency’s perspective, the desired price is p K , for increasing the price 
beyond p R will not cause production to increase and excess demand vanishes at p K . Thus, the 
agency will allow price to increase until excess demand vanishes and market is cleared. 


•See D P Baron, 111. for another approach 
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In fact, the firm has constant marginal cost (and < p R for otherwise the market would not 
clear and the agency would revise its estimate of p R upward). Consequently, the agency eventu¬ 
ally allows the price to reach the monopolistic price. 

Finally, the speed of the price adjustment varies directly with the disequilibrium as meas¬ 
ured by the excess demand x (see assumption 2 below). 

3. ANALYSIS OF MODEL I 

We begin by anlyzing the dependence of the optimal production strategy on the current 
price. Defining / - (0, 1, 2, ... } and n ip) - ip - c)Dip), our results require the following 
assumptions: 

1. kip, x) is a differentiable function of x. 

2. kip, x 2 ) > kip, xj), x 2 ^ x v 

3. A ip, 0) - 0. p € /. 

4. kip + 1. x) < kip, x), p 6 /. 

5. A ip + 1. x 2 ) - A ip + 1. X|) < k ip. x 2 ) - A ip. X|). p € /. x 2 > X|. 

6. A ip + 1. x 2 )/A ip + 1. x^ ^ A ip. x 2 )/k ip, xi), p € /, x 2 > X|. 
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7. D(p + 1) < Dip), p € /. 

8. There is a p € / such that w (p) > »r (p) for all p € /. 

9. nip + 1) - 7r(p) < nip) — nip — 1), p — c f 2.p. 

10. it ip + l) < n ip), p — p, p + 1. 

Assumption 1 is made for the sake of convenience. Assumption 2 reveals that the larger 
the excess demand the greater the tendency for the price to rise. If the likelihood that the 
regulator will grant a price increase increases with the observed level of excess demand then 
Assumption 3 will be true. This is true in the natural gas industry (see [9]). Assumption 3 
states that the firm may maintain the current price if it so desires by producing exactly what the 
consumers demand. Assumption 4 implies that at higher price levels it is more difficult for the 
firm to induce price increases, whereas assumptions 3 and 6 imply that at a higher price an 
increase in unsatisfied demand has a smaller effect on the rate at which the price increases. We 
note that Assumptions 5 and 6 are trivially satisfied if A ip, x) is independent of p. Assumption 
7 merely asserts that the demand curve for the monopolist’s product is downward sloping. 
Assumptions 8-10 are valid for a variety of reasonable demand curves including Dip) — ae bp 
ia, b > 0), Dip) = p~ k ik > 1), and Dip) - a - bp ia, b > 0). Finally, we make the rea¬ 
sonable assumption that whenever the current price is p, the monopolist will never produce at a 
rate exeeding Dip). This is reasonable because any production in excess of Dip) will earn no 
revenues and (by Assumptions 2 and 3) will have no effect on the future price. This assump¬ 
tion (coupled with Assumption 7) allows us to assume that the set of possible actions for each 
price is a finite set. 

We now define Vip) to be the maximum expected discounted profit earned over an 
infinite horizon when the current market price is p. It follows that 

(1) Vip)— max Jip, x), 

where Jip, x) - [ip - c) iDip) - x) + A ip, x) Vip + 1)}/(A ip, x) + a). 

Let x(p) be the largest value of p attaining the maximum in (1). Our goal is to derive condi¬ 
tions which ensure that xip) is a nonincreasing function of p; that is, the higher the price the 
smaller the amount of consumer demand that will be left unsatisfied by the firm. 

Our analysis of the behavior of xip) will be facilitated by the study of a modified version 
of the infinite horizon problem described above. Towards this end we let V„ip) be the max¬ 
imum expected discounted profit earned by the firm over an infinite horizon when the current 
price is p and at most n price increases are permitted. It follows that iV 0 ip) equals n ip)/a if 
p > c and 0 if p < c) 

(2) F„(p) - max J„ip, x) n > 1, 

where J„ip, x) - {ip - c) iDip) - x) + k ip, x) F„_, (p + 1)}/(A ip, x) + «). 
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Our main result will require the following lemmas. The trivial proof of Lemma 1 will be omit¬ 
ted. 


LEMMA I: If Assumptions 8 and 9 are valid, then for « > 0 and c < 
P < p. F,(p) > ir(p - 1 )/a. 

LEMMA 2: If Assumption 10 holds, then for p > p and n > 0 F„(p) — J„{p. 0). 

PROOF: The result is clearly true for n - 0. We therefore assume the result to be valid 
for n - I and verify that it remains valid for n. To prove this observe that 

(a -EX(p. x))J n (p, 0) « (a -EX (p, x))n(p)/a [by (2)1 

> (p - c) ( Dip ) - x) -E X (p. xhrip -E l)/a [by Assumption 10] 

- (a -E X ip, x))J„(p, x) [by the induction hypothesis]. 

Thus, the induction hypothesis implies that J n (p, 0) - n (p)/a so the above inequalities imply 
that J„(p, 0) > J„(p. x) which is the desired result. 

LEMMA 3: If Assumptions 4, 7, 8, 9, and 10 are valid, then for n > 0 and 
c < p < p - 1. 

(3) a V„(p -E 1) — 7r(p) < a V n (p) - it (p - 1). 


PROOF: We prove (3) by induction. For n — 0 (3) reduces to Assumption 9. We 
therefore assume that (3) is valid for n - 1 and verify that it remains valid for n. Define 
x(n. p) to be the largest value of x satisfying V n (p) - J„{p. x(n. pi). Let x - x(n, p -E 1). 
Since Dip -E 1) < D ip), x is a feasible action when the current price is p. This implies that 

(4) a v„(p) > a[(p - c) ( D(p) - x) + X ( p, X) V H -\ip -E 1))/<X (p. x) + a). 

(5) a V n (p + l)«o((p+ 1- c) (Dip + 1) - x) 

+ X(p + 1, x) Y„~](p + 2)}/(X(p + 1, x + a) (by (2)) 

< a[(p -E 1 - c)(D(p -E 1) - x) -EX (p, x) y„-i (p + 2)}/(X (p, x) + a) 
[by Lemma 1 and Assumption 4]. 

Together (4) and (5) imply 

(6) «(Mp + 1) ~ F„(p)) < a Up + 1 - c) (Dip + 1) - x) - (p - c) (Dip) - x) 

+ X(p, x) [K„_,(p + 2)- V H -\(p + l)]J/(X(p, x) + a). 

Note that 

(7) (p -E 1 - c)(D(p El)-x)-(p-c) (Dip) - x) - trip -E 1) - nip) - x 

< it (p) - n (p - 1) [by (9) and p -E 1 < pi. 

Also observe that for c ^ p < p - 1 the induction hypothesis implies that a(V n _ x (p + 
2) - y*-i(p + D) ^ w (p + 1) - ir (p) while Lemma 2 and Assumption 8 imply 

a(V n - X (p -El) - F„_,(p)) - ir(p + 1) - w(p) < ir (p) - ir(p - 1). 
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Thus, for c ^ p ^ p — 1 it follows that 

(8) a(K„_,0> + 2) - V„-y(p+ D) ^n(p+ 1) - n ip) rip)-nip- 1). 
Together (6)-(8) imply that 

aiV n ip + 1) - V n ip)) < ir(p) - nip - 1) c < p < p - 1 
which is the desired result. 

Lemma 2 implies that for p > p the firm's optimal policy is to satisfy all consumer 
demand by producing at a rate Dip). We now show that for n > 0 and c < p < p — 2, 
xin. p) > xin, p + 1). Coupling this result with Lemma 2 we see that for all n and p > c 
the firm’s optimal level of unsatisfied demand (the amount of market disequilibrium) is a 
nonincreasing function of p. 

THEOREM 1: If (l)-(lO) are valid, then for n > 0 and c < p < p - 2. 

(9) xin, p + 1) < xin, p). 

PROOF: Observe that (9) will follow if we can prove that 

(10) J„ip + 1. X 2 ) - J n ip + 1. x^ > 0>^ J n ip, x 2 ) - J„ip. *i) > 0 
holds for n > 0, p + 1 < p — 1, and Dip + 1) > x 2 ^ x,. 

We now proceed to verify (10). After some algebraic manipulations J„ip + 1, x 2 ) — 
J„ip + 1. x,)Si Ocan be shown to be equivalent to 

(11) a V„_!ip + 2) - ir(p + 1) ^ ip + 1 - c){«(x 2 - x,) 

+ X^ ip + 1, X|) — X^ ip + 1, X 2 )}/(A ip + 1, X 2 ) — X ip + 1, X|)). 


By (11), (10) will hold if the left side of (11) is a nonincreasing function of p and the 
right side of (11) is a nondecreasing function of p. Since p + 2 < p. Lemma 3 implies that the 
left side of (11) is nonincreasing in p. Since Assumptions 2 and 5 imply that 
a ip + I - c) (x 2 — X\)/K ip 4* 1. x 2 ) — A ip + 1, X|)) is nondecreasing in p , the right side of 
(11) will be nondecreasing in p if 

x^ ip + 1, X|) - X|A jp + 1, x 2 ) ^ x^ ip, x,) - x,K jp 2 , x 2 ) 

A ip + 1, x 2 ) - A ip + 1, X|) ^ A ip, x 2 ) — X ip, X|) 

Upon simplification (12) is seen to be equivalent to (x 2 - Xi) {A ip + 1, x^kip, x 2 ) - 
Kip + 1, x 2 )A(p, x^} > 0. Since x 2 ^ x ( , the last inequality follows from Assumption 6. 
This completes the verification of (11), and the proof of Theorem 1 is complete. 

The following Lemma relates V„ip) to Vip). 

LEMMA 4: For all p lim,,_ V„ip) “ F(p). 

PROOF: Since nip) is bounded above by n ip), the result follows directly from Lemma 
l of (21. 
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Lemmas 2 and 4 imply that for n ^ p, V n (p) - V{p). Theorem 1 plus Lemmas 2 and 4 
now imply that Vip) - Jip, 0) ip ^ p), and x(p + 1) < x(p), ic < p < p - 2). 

Let x a (p) and U a (p) denote the dependence of these quantities on the discount factor a. 
We now show that x a (p ) is a nonincreasing function of the discount factor a. In order to 
prove this result, we first consider a finite horizon semiMarkov decision process formulation of 
the problem. We use the technique of Lippman [61 in which the system is observed and the 
production rate can be changed only at discrete points in time, namely those corresponding to 
price increases and certain null events. These null events, which do not change the current 
price, are introduced in order to make the time between observations independent of the state 
and action and they render the problem equivalent to a discrete time Markov decision process. 
After analyzing the discrete time Markov process we will let the horizon length go to infinity 
and exploit the equivalence of this formulation to the original continuous time problem to get 
results for our original problem, that is, the inifinite horizon problem in which the production 
rate can be changed at any point in time. 

We now consider the system to be observed only at times when price increases or null 
events are observed: The time between observations is exponentially distributed with parame¬ 
ter A - sup X (p, x)« X (0. Di 0)), where the last equality follows from Assumptions 2 and 

P u 

xiDip) 

4. The state of the system is p, the current price, and the action is x if production is occurring 
at a rate Dip) - x. The probability that the next observation point is occasioned by a price 
increase is X (p. x)/\ while the probability that the next observation point is a null event is 
1 - X (p. x)/\. 

Let W na (p) be the maximum expected profit earned by the firm when n observation 
points remain, the current price is /, and the discount factor is a. Then 

(Wo.a(p)~ 0) 

(13) W na (p) — max G na ip, x)/(A + a), n > 1, 

Oix^Dlp) 

where 

G na ip. x) - piDip) - x) + X (p. x)A M'„_| a ip) + A a (p) 
and 

^.(P + D- K.a (P>- 

Let x„ a (p) be the largest value of x attaining the maximum in (12). We now prove that 
if Assumptions 1, 2, 3, 7, 8, and 10, and nip + 1) > w(p)(p- c, c + 1, ... p), are valid 
then x„ a ip) is a nonincreasing function of a. By Theorem 1 of [5] and Theorem 1.1 of [101 
this will imply that x 02 (p) ^ x 0) (p) (a 2 ^ a t ) which is the desired result. Before proving this 
result we require the following lemma. 

LEMMA 5: If Assumption 10 holds, then for n ^ 0 and p > p, W„, a {p) “ G„ a ip, 0). 

PROOF: The proof is virtually identical to the proof of Lemma 2 and is therefore omit¬ 
ted. 
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LEMMA 6: If Assumptions 1, 2, 3, 7, 8, and 10 are valid and it ip + 1) > ir ip) is valid 
for c < p < p, then A H^ ao (p) >0(c<p^p-l). 

PROOF: We prove the result by induction on n. For n « 1 the result follows immedi¬ 
ately from ir (p + 1) > 7T (p). Assuming the result for « - 1 and letting x — x„„(p + 1). (13) 
and x - Dip + 1) + Dip) < Dip) imply 

(14) (A +a)d\V na ip) > G na ip + 1, x)- G na ip, x - Dip + 1) + Dip)) 

- (A — A (p, x- Dip + 1) + D(p))A IF*- 1 . 0 (p) + A(p + 1, x)A (p + 1). 

For c < p < p - 1 the nonnegativity of the last expression follows from the induction 
hypothesis. To complete the proof it therefore suffices to show that W Ha i~p) > W na ip - 1). 
To prove this, note that nip) > nip - 1) the induction hypothesis, and Lemma 5 imply 

W„ a ip - 1) - ip - c - 1) (D(p - 1) - x H ip - D) 

+ \(p- 1, X„ a iP ~ D) „(p) 

+ (A -Kip - 1, x HOt ip- D) w n _ {a ip- 1))/(A +«) 

< |ir(p) + A ^ n _ l a (p)}/(A + «)- W na ip) 


This completes the proof of Lemma 6. 

We can now prove 

THEOREM 2: If Assumptions 1, 2, 3, 7, 8, and 10 are valid and ir(p + 1) ^ nip) is 

valid for c < p < p - I, then fora 2 > a t and c 4 p ^ p - 1 x H0 , 2 ip) ^ x„ af ip). 

PROOF: By reasoning analogous to that used to justify (10) it suffices to prove that 
G na2 ip. x 2 ) - G n a 2 ip, X,) ^ G„„ ( ip, x 2 ) - G BO| (p. X,) 
holds foraj >a b and x 2 > x ( , and c < p ^ p - 1. By (13) this inequality is equivalent to 

(15) (A ip. x 2 ) - A ip. x,)) (A W„- i ai ip) - A W,_,. B| (p)) < 0, 

By Assumption 2, inequality (15) will hold if 

(16) A F n -i. aj (p) < A aj (p), ft 2 > ai. C < p < p — 1. 

To prove (16) by induction note that for n - 1 (16) follows from 

1/(A +a 2 ) < 1/(A + a,). Assuming that (16) is valid for n - 1 we can verify it for n by 

observing that 

(A + a) H'' fla| (p) > G„_ ai ip + 1, x,) - ( ip, x,) 

and 

(A +a) W„,„ } ip) < G na} (p + \, x,) - G„. aj (p + 1. x 2 ), 
where X! - x,. a (p + 1) and x 2 - x na ip). 
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These inequalities imply that 

(A + a)(A W„ ax (p) - A W Haj (p)\ > {A (p + 1. x,) (A 0| <p + 1) 

- AHV.,. aj (p + D) 

+ (A - A (p, * 2 > <A AH' Jf _ l oj (p»| 

^ 0. 

where the last inequality follows from the induction hypothesis. This completes the proof of 
Theorem 2. By the remarks following Lemma 4 the hypotheses of Theorem 2 also imply 

Xa 2 (P > < *„,</>) 

We now characterize the dependence of x„ a ip) on n. More specifically we show that the 
hypotheses of Theorem 2 imply that x B+ | ,,(/>) > x Ba (p) is valid for c < p < p - 1. If we 
assume (hat the firm wishes to maximize its expected discounted profits over a finite horizon of 
length T < °o and we define x, a ip) to be an optimal action when a time / remains. Theorem 4 
of [6] and the above result will enable us to conclude that x l} „(p)> x f) „(p) for r 2 ^ t\. 

THEOREM 3: If the hypotheses of Theorem 2 are valid, then for n ^ 1 and 

C < P < p ~ 1 X B+ ! „(p) ^ X„„(/>). 

PROOF: The result will follow if we can prove that 

G„ +) .„(a ~ G b+ , a ip. x,) > G„ a (p. x 2 ) - G„„(p. x,) 

holds for n > l, xj > X|, and c < p < p - 1. By (13) this inequality is equivalent to 
(A(p. x 2 )-A (p, X|))(A (p) “ AIF^i^lp)) > 0. By Assumption 2, the last inequality 

will follow if 

(17) A I(p) ^ A )F B _, a (p), « > 1. c < p < p - 1. 

The proof of (17) is similar to the proof of (16), and is therefore omitted. 

3. ANALYSIS OF MODEL 2 

We now characterize the dependence of the optimal production strategy for Model 2 on 
the current price. Our results require the following assumptions: 

11. A (x 2 ) > A (xi), x 2 ^ X| 

12. A (x) is differentiable 

13. A(0)-0 

14. Xj /(A (x 2 ) +a) ^ Xi/(A(X|) + ar), x 2 > x ( 

15. There is a p € / such that nip) ^ w (p) for all p € /. 

16. For p 6 /. 7r(p) is a concave function of p, that is nip + 1) - n(p) ^ nip) - 
nip - 1). 
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Assumption 14 is the only condition which requires any explanation. If X (x) is concave, then 
Assumption 14 will be valid. To show this define A Or) — x/(X (x) +<*). Then for any x 0 > 0 
A'(x 0 ) ” X (xq) +« - (x 0 )/(X (x 0 ) +o) J . Together Assumption 13 and the concavity of 

X (x) imply that X(x 0 ) > X(0) + XoX'(x 0 )« x^'(x 0 ). This shows that h'(x 0 ) > 0, which 
implies Assumption 14. 

We now define Tip) and T„ip) to be the analogs (for Model 2) of V(p) and VJp). 

Then 

(18) Tip) - max Hip. x), 

0 <x<D(p) 

where 

Hip, x) — {(p - c) iDip) — x) + X (x) £ /(A:) r(p + k)tik (x) + a) 

*-i 

and 

(19) T„ip)“ max Hjp, x). 

0 *x<D(p) 

where 

Hjp, x) - lip - c)iDip)~ x) +X(x) *£ fik)T n . x ip + *)}/(X(x) +«). 

We also define xip) and x n ip) to oe the largest values of x attaining the maximum in (18) and 

(19) , respectively. The proof of the following lemma is virtually identical to the proof of 
Lemma 2 and is therefore omitted. 

LEMMA 7: If Assumptions 15 and 16 are valid, then for n > 0 and p > p 
TJp) — HJp. 0). 

As in Section 3, we now focus our attention on showing that for c < p < p — 1, 
x H ip + 1) < x„ip). Before proving this result we require the following Lemma: 

LEMMA 8: If Assumptions 15 and 16 are valid, then for p > c and n > 0. 

(20) a *£ /(*)r„_,(p + 1 + k) - nip + 1) < a £ fik)T n -\ip + k) -n ip). 

k -\ *-1 

PROOF: It sufficies to show that for each k > 1 

(21) «T s _,(p + 1 + k) - nip + 1) < «T,_,(p + k) -nip). 

For n - 1 (21) reduces to Assumption 16. We therefore assume that (21) holds for 
n - 1 and verify that (21) holds for n. Letting x - x„ (p + 1 + X) we see that aT„ip + 
I + k) - HJp + 1 + k.x) and 

a T„ ip + k) > a HJp + k, x). 
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The last two statements imply 

(22) att^p + 1 + k)- T n (p + It)) < «{(p + k + 1 - c)(D(p + k + \) - x) 

(p + k - c) (Dip + k) - x) 

+ A(x) £ f{j)[T n -\(p + k + j + 1) - r„_,(p + k + y'D/ft Or) + <*). 


ip + It + 1 - c) (Dip + k + \) — x) — (p + k - c) (Dip + k) — x) 

— irip + Ar + 1) — rr{£) — x < rr(p + 1) — ir (p) [by Assumption 16) 


a ( T „-j (p + It + j + 1) - T„-\(p + k + j)) < it (p + 1) - n (p), 
[by the induction hypothesis) 

(22) implies that 

a (7"„_| (p + 1 + k) - T„- l (p + k)) ^ ir(p + 1) - ir (p), 
which is equivalent to (21). 

THEOREM 4: For n > 0 and c < p < p - 2 

(23) x„(p + 1) < x„(p). 


PROOF: The proof is similar to the proof of Theorem 1. As before, it suffices to demon¬ 
strate that for Dip + 1) > x 2 > Xj, p < p - 2, and n > 0 

//„(p + 1. x 2 ) - H n (p + 1. x,) > O-* H„(p. x 2 ) - H„(p, x,) > 0. 

To show this, note that H„(p + 1, x 2 ) — H n (p + 1, X)) > 0 may be shown to be equivalent to 

(24) (X (x 2 ) — k (x])) la £ f(k) T„.\(p + k + 1) - ir(p + 1) 

( *-i 

> (p + 1 - c) Ix 2 (\ (xj) 4- a) - X|(X (x 2 ) + a)). 


By Lemma 8 the left side of (24) is a nonincreasing function of p. Since Assumption 14 
implies that the right side of (24) is a nondecreasing function of p it follows that (24) implies 
H n (p. * 2 > ~ H„(p, Xj) > 0. This completes the proof of the theorem. 

As in Section 2, it easily follows that limHVp)- W(p) and that for n ^ p, 

W H (p)“ Wip). Together with Theorem 4 plus Lemma 8, these facts imply that 
x(p) > x(p + 1) (c < p < p - 2) and that in the unmodified infinite horizon version of 
Model 2 the firm should produce at a rate Dip) for p ^ p. 

Unfortunately, all efforts to determine how the optimal production policy for Model 2 
depends on the discount rate and the length of the horizon have been unsuccessful. 
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5. EXTENSIONS AND SUGGESTIONS FOR FUTURE RESEARCH 

One obvious defect of our model is that decreases in price are not allowed. A model 
which allowed for the price to decrease from p to p - 1 in a time governed by a Poisson pro¬ 
cess with rate p ip, x) (with p ip, x 2 ) < p (p, X|)) as well as increase from p to p + 1 in a time 
governed by a Poisson process with rate Kip, x) (Kip. x 2 ) > K(p, *|)) would be of obvious 
interest. Unfortunately, we have been unable to obtain characterizations of the optimal produc¬ 
tion policy for such a model. 
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ABSTRACT 

Moment and maximum likelihood estimates (nvl.c.'st are investigated lor 
nonparametrie and parametric models lor a single server queue observed over a 
random lime horizon, namely, up to the nth departure epoch Also, m I e \ of 
ihe mean interarriv.il time and mean service time in an U.'MH queue ob¬ 
served over a lived lime-interval are vtudied Limit distributions of these esti¬ 
mates are obtained without imposing steady slate assumptions on the queue- 
size or waning lime processes 


I. INTRODUCTION 

The theory of queues as developed so far is largely a descriptive theory, namely, it is con¬ 
cerned with the probabilistic structure of the models and the behaviour in finite time / as well 
as in the limit as t — °° of the processes arising from these models. Relatively less has been 
done on a prescriptive theory dealing with the statistical analysis, design and control of queue¬ 
ing systems. Yet from the practical point of view these latte - aspects are very important. Thus, 
for example, the management of a service facility subject to congestion and wishing to design 
an efficient queueing system should be in a position to estimate the various parameters of the 
model on the basis of data collected at the facility. 

The earliest paper on problems of statistical inference from queueing models seems to be 
that of Clark I4J who investigated the problem of maximum likelihood estimation of the param¬ 
eters of an M/M/\ queue in equilibrium. While this paper is of historic importance, the more 
substantial investigation of these problems was carried out by Cox [5] and Wolff [14]. The 
work of Cox contains several ideas, but no major results seem to emerge. Wolff discussed 
maximum likelihood estimation and likelihood ratio tests for a class of ergodic queueing models 
which give rise to birth and death processes (including the queue-length process in Ml M/s and 
related systems). Since these processes are continuous time Markov processes, Wolff was able 
to derive limit distributions of the estimates and test statistics as a direct application of 
Billingsley's [3] results for Markov processes. The Markov properly of the queue-length pro¬ 
cess and the assumption of steady state are the essential features in Wolff's study as they are in 
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Billingsley’s. For further work on the M/M/X queue see Harris 18], Jenkins (10], and Samaan 
and Tracy [13], Benes 12] treated the estimation problem for the M/M/oo system. Goyal and 
Harris [7] obtained maximum likelihood estimates of the parameters of a queueing system with 
Poisson arrivals and state-dependent service, again assuming steady state. Henningsen [9] 
investigated the M/G/ 1 queue in equilibrium and obtained estimates for the arrival rate and 
the parameters of the service time density. Crane and Lemoine (6] have applied simulation 
techniques to the problem of estimating the steady state mean waiting time in a single server 
queue. In their recent monograph, Basawa and Prakasa Rao (1] discuss examples of inference 
for systems such as M/E k /\ and EJM/X, again within the framework of Billingsley (3]. 

The objective of this paper is to consider a single server queueing model and derive ‘rea¬ 
sonable’ estimates for the interarrival and service time distribution functions (d.f.’s) and their 
means. The processes arising from this model are not always Markovian, nor do we assume the 
existence of steady state. Our estimates are not necessarily maximum likelihood estimates, but 
have a simple structure of intuitive appeal and reduce to m.I.e.’s in special cases. The estimates 
of the means are moment estimates and those of the d.f.’s are empirical d.f.’s. Limit distribu¬ 
tions of the estimates are obtained using direct and simple arguments. 

In Section 2 we study the properties of moment estimates of the interarrival and service 
time means, and also the estimates of the corresponding d.f.’s in a G/G/ 1 queueing system 
observed over a random time horizon (0, D„l, where D„ is the nth departure epoch. The limit 
distributions of these estimates are obtained without imposing any restrictions on the traffic 
intensity. Section 3 is concerned with the maximum likelihood estimation of the parameters of 
a G/G/ 1 queue, using the same sampling plan as in Section 2. In Section 4 we discuss m.I.e.’s 
of the interarrival and service time means in an M/M/\ queue observed over a nonrandom 
time-interval (0. /), and study their properties as t -* «>, again without any restrictions on the 
traffic intensity. The problem of hypotheses testing will be considered in a forthcoming paper. 

NOTATION: We shall use to denote convergence in distribution. The normal ran¬ 
dom variable with mean p and variance o- 2 will be denoted by N\(p. <r 2 ), and the random vec¬ 
tor having the bivariate normal density with means mu/* 2 and variance-covariance matrix £ will 

be denoted by W 2 | J^J, I .As usual, a.s. means almost surely. 

2. THE G/G/l QUEUE-MOMENT ESTIMATES 

Consider a single server queueing system in which the interarrival times [u k , k > 1} and 
the service times {v*. k ^ 1) are two independent sequences of independent and identically 
distributed nonnegative random variables with d.f.’s F and G respectively. Assume that the 
moments 

(1) E(u k ) “ a, Var(«*) - <r / 

(2) Ei\ k ) * b, Varfv*) = <r 3 2 

are all finite. The traffic intensity of the system is then p — b/a\ also, let it “ maxd, p). We 
assume that the initial customer arrives at i - 0. Our sampling scheme is to observe the 
phenomenon until the first n customers have departed from the system and note the service 
times of these n customers, say (v ( , v 2 , .... v„). Let the nth departure epoch be D„ , and also 
observe the interarrival times of all customers who arrive during (0. £>„]; thus, we obtain 
(u,, u 2 .« v ). where 
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(3) N ,4 = N A ( D „) - max [k: u\ + u 7 + ... + u k < £),,}. 


Clearly, N A > n. For the means a. b we propose the estimates 


(4) 


TT 1 

^ i 


n 


Iv, 


it should be observed that the estimate b„ is the usual sample mean, whereas a„ is based on a 
random number of observations. It would be interesting to compare a„ with the ordinary sam¬ 
ple mean 


(5) >;• 

n I 

It turns out that asymptotically a„ and a* display similar behaviour, namely, they are both con¬ 
sistent estimates of a and asymptotically normal, with a„ having a smaller variance. The obvi¬ 
ous independence of a* and b„ is used to establish the asymptotic independence of the esti¬ 
mates (4). These properties are proved in Theorem 1 below. We need the following prelim¬ 
inary result, which will be proved in the appendix. 


LEMMA 1: As /i — 

( 6 ) — N a —‘ 7] in probability. 

n 

THEOREM 1: As n — °°, 


(i) a„ — a, b„ —* Aa.s., and 



Vn (a„ — a) 


0 


WAi 0 n 

(ii) 

\fn ( b„ — b) 

N, 

0 

' 

o wll 


PROOF: (i) The strong law of large numbers gives 

(7) -j- 1 *■ ^ I v , — b a s ' 

Since ( 6 ) implies that N A (D„) \ °° as n — we also obtain 

(8) TT L «i ~ a a s - 
n a | 

as n — We have thus proved both statements in (i). 

(ii) We first consider the limit distribution of Vn (oj - a), -Jn (b„ - b ), where a* is the 
sample mean (3). On account of the independence of the sequences (a*) and {vj we have 



Vn(a*~ a) | |o| [o' i 2 0 | 

(9) 

^^114 (0 W| 


by the central limit theorem. The difficulty is that our estimate a„ is based on N A observations 
where (3) shows that N A depends on D„ and hence on {vj. However, we shall show that this 
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dependence of a„ and b„ disappears in the limit as n —♦«>. The desired result (ii) will be 
proved via the Cramer-Wold device, if we show that for arbitrary real numbers a, fi, 

(10) ayfri (a„ - a) + fiy/n (b„ - b) N, jo, a 2 —— + /3 2 o- 2 2 j. 


To this end, we write the left side of (10) as 


a 

\ 1/2 

— (a* - a) + fi-Jn ( b„ - b) 

+ a 

yfn ( a„ - a) — 


1/2 

(a* - a) 


V 



V 



The limit distribution of the term within the first pair of brackets in (11) is given by (10) on 
account of (9). Therefore, the desired result (10) will follow if we show that the term within 
the second pair of brackets in (11) converges to zero in probability. Now 



1 1 1/2 

1 »1/2 


1/2 

(12) 

An (a„ — a) - — (oj- a) ” 

U- - 




hi 

Kl 

V 



1 

M' 2 


I *«/" o) —T777 S- ( "> ~ a) 


First, we note that by the Central Limit theorem applicable to random sums (see Billingsley 
[3]) it follows, in view of (6), that as n — <», 

<>3) ~~ 1 (u, ~ a)-*> \ { (0, ,2) 

N A I 

and that 

v 

(,4) “T7T “777 ib*,-a)~0 

"A t n I 

in probability. Using (13), (14) and (6) we see that the right side of (12) converges to zero in 
probability as /»—«>. Thus, (10) is proved and the proof of Theorem 1 is complete. 


For the d.f.'s Fand G the natural estimates are the empirical d.f.’s 

(15) F n (u) - £6 <m - Ui ), G„(v>« - fs(v- v,) 

n a i n | 

where 8(x) - 0 for x < 0 and - 1 for x > 0. The asymptotic properties of these estimates 
are given by the following theorem. Its proof is similar to that of Theorem 1 and is therefore 
omitted. 


THEOREM 2: For all u > 0, v > 0, we have as n 
(i) F„(u) —• F(u), G„(v) —• G(v) in probability, and 


(ii) 


V^(F„(w) - F(u)) 

y/H(GA\) - G(v)) 


/V, 


I:]. I; 2 " °J 
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where 

(16) F(u) (1 - F(u)], o 2 - G(v)[l - G(v)]. 

3. THE G/G/l QUEUE-MAXIMUM LIKELIHOOD ESTIMATES 

In this section we assume that the d.f.'s F and G of the interarrival times and service 
times are absolutely continuous with continuous densities f(u ; 9) and #(v; 0), where /and g 
are known functions of unknown real parameters 9 and 0. For simplicity of presentation we 
assume that 9 and 0 are scalars; the case of vector parameters can be treated in an analogous 
manner. Consider the problem of maximum likelihood estimation of 9 and 0. Under the sam¬ 
pling scheme of Section 2 the likelihood function is given by 

; 0)| [1 - F(X n -9) 1 

»4 

where X„ = X„(D n ) »* D„ - £ u r The only factor that causes any difficulty (in the sense of 

i 

yielding simple estimates) in (17) is 1 - F(X„\9) S which corresponds to the incomplete arrival 
interval when sampling is terminated at the epoch D„. Consider instead 


L„if. g) 


H|n 


(18) 


L°Af. g ) 



which can be viewed as an approximation to L n (f g) in a sense to be explained later. Let 9 ° 
and 0 “ denote the likelihood equation estimators (see Rao [12]) of 9 and 0 based on L“(/, #); 
thus#", 0 ° are the roots of the equations 

a « a 

09) £ -£r log /(«/, 9) - 0, £ — log g(v ( ; 0) - 0. 

| O" | O0 

The following theorem states the asymptotic properties of 9 ", 0 


THEOREM 3: Under appropriate regularity conditions of /and g (see Rao [12], for 


example) we have as n — °° 

(i) 9 " — 9, 0 " —> 0 in probability, and 
yfn (fl " - 9) | |0 |cr# 2 /i) 

\fit (0 " — 0 ) j ^ 2 | 0 [o 

where 

(20) <r# 2 - ^ 3^ ,0 8 





PROOF: The stated properties of 0; follow from the classical maximum likelihood theory 
(Rao [12]), while the results for 9° can be deduced from the classical results using the strong 
Law of Large Numbers and Central limit theorem for random sum involved, namely, as 
n — <®, 


VOL. 28, NO 3. SEPTEMBER 1981 


NAVAL RESEARCH LOGISTICS QUARTERLY 











480 


I V BASAWA AND N U. PRABHU 


~N~ ^ 0 *n probability 

<v 

>ob/(m,;0 ) AMo. o-„- 3 ). 

The use of Lemma 1 then allows us to replace the random scale N 4 by the usual scale n to get 
the desired properties of 9 The asymptotic independence of 9° and 4> a „ can be verified via the 
Cramer-Wold device as in the proof of Theorem 1. Details are omitted. 

Returning now to the full likelihood function (17), let us denote by and <£ „ the likeli¬ 
hood equation estimators based on it; thus, 9 „ and <£„ satisfy the equations 

V| a 

(21) Z ~ log/(u/,fl) +//U n ;fl) = 0 

i do 

( 22 ) Z ~ir lo 8 £( y /'^ ” 0 

i o<f> 


(23) H{x\0) = ~ log (1 - F(x\9)). 

<jv 

Comparing (21)-(22) with (19) it is seen that <£„ *» <£“, while 9„ differs from 0". We seek con¬ 
ditions under which the estimates 9 n and 9“ are asymptotically equivalent. For the purpose of 
motivation we first consider the important special case of Poisson arrivals. We have the follow¬ 
ing: 

THEOREM 4: For the M/G /1 queue with mean interarrival time® and service lime den¬ 
sity #(v; d>) we have 

D„ - 1 v < 

(i) 


(ii) yfn (9„ - 9) and %/n (9% - 9) have the same limit distribution, namely TV,(0, 9 1 h\). 

PROOF, (i) We have /(u;0) - 9-'e~ u '\ so that 1 - F(u\9) - and 
H(x\9) - x9~ 2 . 

The first of the Equations (19) reduces to 

-£ + ;r 4 

while (21) reduces to 

N 4 l ^ X n 
9 9 2 7 ' 0 2 

These equations yield the estimates 0% and 9„, respectively, as given by (i). 


(ii) In view of Theorem 3 (ii) it suffices to show that as n — °° 
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Under the condition (24) we are now able to establish the desired asymptotic equivalence 
of the estimates#,, andfl“. We have the following: 

THEOREM 5: Under the condition (24), 7n (#„ - 9) and 7n (# “ - 9) have the same 
limit distribution. 


PROOF: We first approximate 


77 w log LM 8) and 77 Jr 108 8) 

by linear functions of 7n (9,, -9 ) and 7n(9“-9) in the usual manner (Rao [12] and then 
use condition (24), namely. 


(24') 


1 

yfH 


Jj- log L„if g) - log /.,?(/. g ) 


0 


in probability. 


4. THE M/M/1 QUEUE - m.l.e.’s BASED ON A SAMPLE 
FUNCTION OBSERVED OVER A FIXED INTERVAL (0, tl 


In Sections 2 and 3 we observed the queueing phenomenon over a random time horizon 
(0. D„), where D„ is the nth departure epoch. This choice of observation period resulted in a 
considerable simplification in the forms of the estimates and their limit distributions. Corollary 
1 shows that in the case of the Ml Ml 1 queue the moment estimates coincide with the approxi¬ 
mate m.l.e.’s of the interarrival and service time means, these latter being asymptotically 
equivalent to the full m.l.e.'s. However, due to the simplicity of the M/Ml\ model we can 
consider a likelihood function based on a continuous observation of the phenomenon over a 
fixed interval (0, r] and study the asymptotic properties of the m.l.e.’s as t — °°. This latter 
sampling plan was used by Wolff [14] who derived the asymptotic distributions for the case 
p < 1 directly from the known results of Billingsley [3] for Markov processes. In this section 
we obtain the limit distributions of the m.l.e.’s without any restriction on p, using elementary 
methods similar to those used in Section 2. 


Let Qit) be the number of customers present in the system (including the one being 
served, if any) at time t. We formulate a sample function representation for Qit) as follows. 
Let Ait) be the number of arrivals during a time-interval (0, fl; then Ait) is a Poisson pro¬ 
cess with parameter X. Let Dit) be a Poisson process with parameter p and independent of 
Ait). Then clearly 

(25) Qit) - Q(0) + Ait) — J o ' 1 0(J _)> 0 dDis) 

where 1 A is the indicator function of the even tA. Here the integral represents Dit), the 
number of departures from the system during (0. M; thus, 

(26) Dit) - J* 0 * 1q(j-) >0 dDis). 

The total time during (0, /] that the server was busy is given by 

(27) Bit) — lp(i-)>o ds. 
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The interarrival and service time means are given by a - X ' 1 and b — p -1 , respectively. The 
traffic intensity is p - X/p. Also, we denote | - min(l, p). We need the following prelim¬ 
inary results, which will be proved in the appendix. 

LEMMA 2: As / — « 


(i) 


DU) 

i 


(ii) 


Bit) 

t 


nt a.s. 

| in probability. 


Since {(?(/), t > 0} is a Markov process with transition intensities 
(/' > 0), = A -1 (/ > 1) 

it follows from Billingsley 13] that the likelihood function based on the sample function (@(s), 
0 < s < /} is given by 

(28) L,ia, b) oc a - AU) e-' ,a b~ 3u) e ~ BU)lb . 

The m.l.e.’s of a and b obtained from (28) are seen to be 


(29) 


a, 


A it) 

We have the following. 


t anr1 t _ Bit) 
,ad t,- m 


THEOREM 6: As / — oo. 


(i) a, — a a.s., and b, b in probability, 


(ii) 


V7 ia, - a) 

V7 (b, - b) 


0 



PROOF: (i) By the strong law of large numbers applied to A it) we obtain 
a, — k~' — a a.s. 


Lemma 2 yields 

f, gd)/r 

Dit)/t 


jL. J_ 


b 


in probability. 


(ii) We write 

(30) Vr (a, - a) 


X f — A it) kt 

a ‘ VxT * Mi)’ 


Here A i t)/t —* X a.s. by the strong Law of Large Numbers. Also, as t —•> » the distribution of 
A it) is asymptotically normal with mean kt and variance kt. Therefore, the right side of (30) 
converges to /V, ( 0 , a 3 ) in distribution. 
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Next consider 
(31) St (b, - b) 


St 


BU) - bDU) 
DU) 

D(f)+I 

T (v,-4) 


St 


-V7 


(v/ - 6) 


D(f) " DU) 
where v,' is the residual service time of the customer in service (if any) at time t. Due to the 
lack of memory property of the exponential density, the distribution of v,' is free of i. There¬ 
fore, writing 




v,'- b 
St 


DU) DU) 

and using Lemma 2(0 we see that the second term on the right side of (31) converges to zero 
as r —’ oo. The first term can be written as 


(32) 


■J 


DU) + 1 


1 


D(,)+l 


DU) "V DU) VD(r) +1 r 

By the central limit theorem applied to random sums we obtain 
1 


(v, - b). 


D (/)+1 

I <V, - b) 


N,(0, b 2 ). 


y/DU) + 1 

Again, using Lemma 2(0, we find that (32) converges in distribution to 


1 


N\ (0, b 2 ) - N,(0, b 3 /t). 


Sib, - b) N,(0, b 3 /t). 


Thus, 

(33) 

The asymptotic independence of a, and b, is not immediately evident because of the depen¬ 
dence of DU) on both the sequences [u k ] and {vj. However, this independence can be esta¬ 
blished as in the proof of Theorem 1 (ii) by first considering the random variable 


(34) 


*r- 


1 


DU) 


O(r) 

I V ' 
1 


which is independent of a,, since the processes A (/) and DU) are independent. We have 
St (b*- b) N,(0, b 3 ) as before, and 


(35) 


St ( a, - a) 0 a 3 0 

Stti:-b) " 2 0’ o 4 


Using (35) we shall prove that 

(36) aSt ( a, - a) +pSt (b, - b) -► N,(0, a 2 a 3 + 0 2 b 3 /t) 


for arbitrary real numbers a, (3. The desired result (ii) will then follow via the Cramir-Wold 
device. 
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Now the left side of (36) can be written as 

07) ja>//(fl, - a) + 0 y- (b*- *)J /sJ>/7 (*, - b) - (b*- b)J. 

The limit distribution of ther term within the first pair of brackets in (37) is 
N i(0, a 2 a 3 + /J 2 b 3 /£) on account of (35). The term within the second pair of brackets in (37) 
can be written as 



<*.-») +Vjf 1 " Vo(7)»| TOT ? 
<V| ~ w ~ D £ <V| " 4) ) 


(v, - b) 


where we have used the fact that V, — b plays the same role as v, - b (1 < i < Z>(/))—see 
comments following (31). Using arguments similar to the ones used in the proof of Theorem 
l(i) we find that all three terms in the last expression converges to zero in probability. This 
completes the proof. 


5. APPENDIX 


Only the outlines of the proofs are given below; for details see the monograph by Prabhu 

[ 111 . 


Lemma 1 is concerned with a single server queue, with notations as in Section 2, while 
Lemma 2 is concerned with the M/M/l queue described in Section 4. 


PROOF OF LEMMA 1: Let W n be the nth customer’s waiting time, and /„ the idle time 
(if any) preceding this customer. Then 

(38) W n+X - max (0, W„ + X„ +1 ), /„ +1 - - min (0, W n + X„ +l ) 

where X k - v* - u k (k > 1). Let S 0 = 0, S„ - X\ + X 2 + ... + X„ (n > 1). Then since 
W 0 — 0, as has been assumed, we obtain from (38) 

(39) W„ - S„ + /„-/,+ l 2 + ... + /„- - min S k . 

O^k^n 

It is clear that the successive departure epochs D„ are given by 

£>i - V|, D„ - u t + «j + ... + u„_i + IF„_, + v„(n > 2). 

In view of (39) we can write 


(40) D„ - v, 4- Vj + ... + v„ + (n > 1). 


Now it is known that (i) if p > 1, then $ n — ^a.s., while (ii) if p < 1, 
— max(0, a - b). Therefore, 


(41) 


A, _ V| 4- V; + ... + v„ i-i 
n n n 


oo a.s., but n _l 


— b + max ( 0 , a - b) - max (a, b). 
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Next let S A (t) — max (k u t + h 3 + ... + a* < /}. Then by the elementary renewal theorem, 
r'N A {t) — a" 1 a.s. Since D„ — <» by (41), we obtain 


(42) 


w 

A, 


a.s. 


From (41) and (42) it follows that 


(43) 


Na . W . A, 

n D„ n 


a 1 max (a, b) 




as required. 

PROOF OF LEMMA 2: (i) We can write 

(44) DU) -DU)- J Q ' l(,< s -,-o dD(s) 
where 

(45) J* o lo ( j-)-o dD(s) — - m(t) — - ^mf lA (s) - ZHs)] 
and it is known that 

(46) — — max (0, p - X) a.s. 

t 

Therefore, 

DU) D(t) , m(r) 

(47) —-— — —--1- —-— —* p — max (0, p — k) — p£ a.s. 

(ii) We can write Bit) - t - lit), where lit) is the idel time during (0, r] and it is known 
that 

(48) -yy—• max (0, 1 - p) in probability. 

Therefore, 

(49) - 1 - - 1 - max (0, 1 - p) - (. 

t t 
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ABSTRACT 


A method is developed for determining the opnm.il policy for entry of cus¬ 
tomers from many independent classes of Poisson arrivals o a lirsi-come. 
lirst-serve (for customers admitted to the queue) single-server queue with ex¬ 
ponential service times The solution technique utilizes a semi-Markov formu¬ 
lation of the decision problem 

INTRODUCTION 

A small but substantial collection of literature has developed on optimal control of entry 
to a queue starting with Naor’s work (10i in 1969 and continuing to the present. Most of this 
work has been concerned with optimal policies and their properties for queues serving a single 
class of customers. In these models the reward and cost structure is the same for each custo¬ 
mer although a customer’s reward may be the value of a random variable. 

In this paper, a model is considered in which several classes of customers arrive in 
independent Poisson streams. Each class has its own reward structure in that each member of 
class m receives a reward R m for service and pays C,„ per unit time spent in the system. This 
generalization has obvious applications in queueing systems. For example, consider an airport 
serving many classes of commercial aircraft (jumbo jets, wide-bodies, four engine jets, three 
engine jets, two engine jets) in which each class has its own reward for service and cost of wait¬ 
ing. It does not appear that the optimal control policy for such cases has been determined. 

Two types of optimum control policies, the individual optimum and the social optimum, 
are considered in this paper. For an individual optimum policy, join or balk decisions are based 
on the expected gain of each individual customer at the time of his arrival. If his expected gain 
is nonnegative he joins; otherwise he balks. Under a social optimum policy, the join or balk 
decision is made based on maximizing the expected gain per unit time considering arrivals from 
all classes of customers. 
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A review of the literature indicates that "control-limit'' policies are optimal for a single 
class of customers. This type of policy specifies that a customer be admitted if the number of 
customers already in the system is less than or equal to a given threshold value (the "control 
limit"). If the control limit is exceeded, the customer is turned away. Naor assumed that a 
control-limit policy is optimal for a single class of customers and then compared the control 
limits for individual and social optima. Yechiali 113] proved that a control-limit policy is indeed 
individually and socially optimum for a GH Ml 1 queueing system serving a single class of cus¬ 
tomers. Subsequent work by several authors ;for example, see 16], [7], 18], [9], [12], and 
[14]) demonstrated that control-limit policies zre optimal for single class models under even 
more general assumptions. 

When considering several classes of customers, the argument that the individual optimum 
policy is a control-limit policy carries over directly, as shown later, from the argument for a sin¬ 
gle class model. However, it is not possible to carry over Yechiali's argument for the social 
optimum policy and a different approach is used. 

Previous work with several classes of customers appears to be quite limited. Balachandran 
and Schaefer [1, 2] deal with several classes of arrivals to M/Gl 1 queues. However, each class 
has the ability to adjust its arrival rate so that the class with the most favorable net benefit 
(reward minus expected cost) dominates the arrivals to the exclusion of all other classes of cus¬ 
tomers. Furthermore, this work is based entirely on long term expected values and no custo¬ 
mers are able to take advantage of short-lived phenomena, e g., a customer who does not 
belong to the single class which dominates arriving to find the system empty will be denied ser¬ 
vice. Edelson and Hildebrand [4] come closer to the problem dealt with in this paper. They 
consider the Ml Ml l model for classes with different reward and cost parameters and balking. 
The claim to have considered more than two classes of customers, but no results for more than 
two classes are presented, and in what seems to be a contradiction they indicate in their conclu¬ 
sion that the solution procedure they have used (which is not specified in their paper) does not 
extend to cases where "dichotomous classification of customers is not acceptable." While they 
are interested in comparing revenue maximizing policies with the social optimum, it should be 
noted that the solution given for the two class social optimum policy in their first example, after 
appropriate adjustments for different definitions, was checked and found to yield the same pol¬ 
icy that would be obtained by the techniques established in this paper. Furthermore, this check 
provides an indication that the general techniques set forth in this paper are, in addition to 
being workable, correct. 

In this paper, after defining the model, it will be shown that Naor's solution for the indi¬ 
vidual optimum for a single class of customers easily extends to M classes of customers. The 
extension of his social optimum solution to many classes of customers is then made by adapting 
the semi-Markov decision process technique used by Yechiali for a single class of customers. 
The optimal policy is shown to be a control-limit policy and simple bounds on this policy are 
presented. An example of the application of the technique to three classes of customers is 
given. 

THE MODEL 

M classes of customers are considered. Customers of class m, m — 1, 2. M, arrive 

in a Poisson stream with mean rate k m . The service times of the single server are independent, 
identically distributed, exponential random variables with mean l/7i. Thus, the model 
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represents an Ml MIX queueing system with customers from M separate groups or classes. 
Each class has its own reward and cost structure. Once a customer enters the system, service is 
first-come, first-served, regardless of class. A member of class m receives a reward for service, 
R„, and pays C m per unit time spent in the system. R m is required to be greater than or equal 
to Cjn for all m. (Customers from any class for which this is not true would balk under all 
circumstances.) For determining the individual optimal policy, each customer is assumed to be 
able to determine the number of customers in the system at the time of his arrival. Also, for 
determining the social optimal policy, an administrator is assumed who knows the number of 
customers in the system at the time of each arrival. An optimal policy is sought; that is, a set 
of join or balk decisions is sought for each class for each possible state (the total number of 
customers in the system) to maximize (i) the expected return to the self-optimizing (indivi¬ 
dual) customer or (ii) the gain per unit time of all customers (social optimum). 

SOLUTION FOR THE INDIVIDUAL OPTIMUM 

If an arrival from class m finds i customers ahead of him, his expected net benefit for 
joining the system is R„ - (/ + 1) Since the net benefit for balking is zero, the custo¬ 

mer will join the system if the number of customers in the system is less than where is 
such that 

(1) R m - + 1) CJ/x < 0 < R m - n Sm CJfi. 

(Customers are assumed to join the system if their expected net benefit for joining is zero.) 
Thus, self-optimizing customers of class m determine a balking point w, such that 

( 2 ) 

where the brackets indicate the greatest integer function. (2) is a straight-forward extension of 
Naor’s result for a single class of customers. 

SOLUTION FOR THE SOCIAL OPTIMUM 

For the social optimum problem, an administrator decides whether or not a customer of 
class m can join when i customers are ahead of him. The sum of the expected net benefits per 
unit time of all arrivals of all classes is shown to be maximized by a policy that imposes a vector 

« n of forced balking points on the customers. n 0 “ (n 0) , n 0j .tin. )» where a member of 

class m is allowed to join if the state of the system is less than w 0 „, but must balk otherwise. 

Let n - (/» t , n 2 . n M ) be a vector of balking points. The sum of the expected net 

benefits per unit time of all arrivals when the balking points given by ware chosen, is given by 

(3) *(«)• £ (h)R m - £ C m L m (n). 

m»l m™ 1 

A^, (n) and L m (n) are, respectively, the effective arrival rate for class m and the contribution of 
class m to the expected number of customers in the system when balking points « are 
employed. A^ (n) and L„ (n) depend on n because the stationary probability that i customers 
are in the system, <M«), depends on h. 

Rue Ill] shows that the following semi-Markov decision process formulation of g(n) is 
equivalent to (3): 
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(4) g(n) - £ <£,(«> X k «>^ K m\Rm - (' + 1) 

1-0 m-l 

where n* - max {n,„i, <£,(”) is the stationary probability that / customers are in the system 

m 

given that the balking points n are employed, and 

1 if / < n,„ 

Ar '" (,) " 0 if » > n m 

(It may be helpful to note at this point that the term k m [R,„ - (/ + \)Cjti) is the gain rate in 
state i due to class m customers joining the queue.) The work of Derman (3] can be used to 
limit the semi-Markov decision process formulation to nonrandomized policies as in (4). Policy 
iteration can be used to obtain from (4) the maximum gain rate and socially optimal joining 
policy. 

Theorem 1 justifies restricting the consideration of optimal policies to control-limit poli¬ 
cies as in (3). However, lemma 1 must be proved first. 

LEMMA I: The social optimum policy will not allow reneging. 

PROOF: Suppose customer A of class m arrives at time T A and joins the system. Later, 
at time T D , customer A departs the system before he is served. The actions of customer A 
affect no customer who arrived before him. The contribution of customer A to the net gain of 
the system over the interval I T A . T 0 ] is -C m (T»- T 4 ) < 0, his holding cost for the time he 
is in the system. If no other customer arrives before customer A departs, his actions have no 
effect on those customers arriving after him. If other customers arrive during \T 4 , T p ), the 
only effect of customer A’s temporary presence in the system is to possibly cause a customer to 
balk when, in fact, without the presence of A this customer would be profitable (and thus join 
the system). Thus, the contribution of customer A to the system is negative when compared 
with his not joining the system at all. 

THEOREM 1: A control-limit policy is optimal for each class. 

PROOF: Suppose that there exists a class m and a state / such that in the optimal policy, 
p, k m (i) 1, but k m (i - 1) — 0; that is, the optimal policy for class m is not a control-limit 
policy. The optimal gain rate is denoted by g*. If the stationary probability of the system occu¬ 
pying state i under policy p, <t> f is equal to zero, then a policy which is the same as the optimal 
except that k m {i) — 0 also yields g*. To prove Theorem 1 for the case <t>C & 0, consider the 
following modification to the model. At the completion of a service, each customer returns his 
expected net benefit for recalculation based on the current slate of the system. Likewise, his 
occupation costs incurred during the service just completed are also returned. The customers 
remain in the order they arrived, but the administrator recomputes the expected net benefit of 
each customer based on the number now ahead of that customer and uses the given policy, p, 
to decide whether or not each customer can re-enter (stay in) the system. Because there is no 
discounting, the gain rate of this model is equivalent to that of the original model for the same 
policy. Since the transitions are Markovian, all relevant information about the future of the 
system is contained in the current state of the system. (The state of the system must give the 
position of every customer to accommodate the new method of assessing costs.) A customer of 
class m who had joined when i customers were ahead of him would be forced to depart when 
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the number of customers ahead of him dropped to i — 1 , since k m (i - 1 ) - 0. Thus, the cus¬ 
tomer would be forced to renege. By Lemma 1, the given policy cannot be optimal for the 
modified or original model. 

Since the expected net benefit for members of any class joining the system decreases as 
the number of customers in the system increases, the optimality of a control limit policy is a 
reasonable result. 

It is no.*' shown (Theorem 2) that for each class m, »o < /?, , where n a is the socially 
optimal balking point for class m customers and », is the individual optimum balking point for 
class m. Thus, «•- max {n, ) can serve as a bound on the state space required for the policy 
iteration solution of (4). 

THEOREM 2. For each class m, //„^ < n 

PROOF: First, the decision made by the administrator for a given arrival does not affect 
the time of arrival of any customer yet to arrive. As previously shown, n s satisfies (1) 

R m — (w, ^ + 1) Cjp. < 0 < R m — n, n C m /n. 

Suppose a customer, customer A , of class m arrives to find the slate of the system i > n s . Let 
T a be the lime of arrival of customer A , and T B be the time of arrival of the next customer, 
customer B. In view of (I), let a < 0 be the expected net benefit of customer A joining the 
system. If all expected costs and rewards are assigned to a customer upon arrival (as in (4)), 
the contribution of the interval [T 4 . T H 1 to the expected net gain is a < 0 if customer A is 
allowed to join. The decision regarding customer A does not affect the expected net gain of 
customers who arrived before him, but it does affect the expected net gain of those arriving 
after him. If customer A does not join, the state of the system found by customer B and all 
others after him is less than or equal to the state of the system if customer A joins. Since 
R m - (/ + 1) C,Jfi , the expected gain for a customer of class m joining when i are in the sys¬ 
tem, is a strictly decreasing function of /', the contribution to the expected net gain of the inter¬ 
val iTfl. 7V1 for any T ( > T 8 is at least as large when customer A balks as when he joins. 
Thus, if i > n, , forcing a customer of class m to balk yields a larger expected net gain or gain 
rate than allowing him to join. 

The two theorems proved above make it possible to solve for the social optimum policy 
using policy iteration or linear programming. For M classes of customers, we seek to determine 
the h which maximizes the gain rate given in Equation (4). The solution technique will be 
illustrated by an example. Suppose that three classes of customers arrive at a single exponential 
server who has a service rate capability of four customers per unit time. The reward for ser¬ 
vice, cost per unit time in the system, and mean arrival rate of each class are given in Table 1. 

TABLE I — Mode! Parameters for the 
Three-Class Example 


Class 


c m 

A m 

1 

3 

4 

‘ 2 

2 

2 

3 

4 

3 

4 

7 

6 
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To see what gin) looks like for a particular policy n , consider g(n<) where n s is the indi¬ 
vidual optimum policy. From (2) 

n s — (n V| , n t2 , n M ) — (3, 2. 2). 

Since max () - 3 at most states 0 through 3 are required to evaluate gin) for any «, 
e.g., 

g(3, 2, 2) = giJi s ) — £ <t>,in,)\ i |/?i - (/' + l)C|//i} 

/-o 

+ £<M« S ) £ x m |/? m - (/ + dc w /m1 

>-0 m-2 

where the <f>,(/t s ), / — 0, 1. 2, 3 can be obtained from the simultaneous solution of the three 
rate equations 

m- l 

3 3 

£x m +/i 0|(n 5 )"« 

3 

(X, + ti)<ft 2 in,) = £ X m 0|(«,) ii<t> } inj 

m- I 

and the normalizing equation 

£ <*>,(«,) ~ *• 

i-0 

The first iteration of Howard's [5] policy iteration algorithm yields the individually optimal 
policy, n,. Because the relative values of occupying the various states (Howard’s v/s) are all set 
to zero, a social optimizing customer cannot determine the effect his joining has on later arriv¬ 
ing customers and, thus, can do no better than the individually optimal policy. Thus, the first 
iteration yields a control-limit policy with balking points n — (3, 2, 2) and the expected gain 
rate associated with the policy is g = 2.486. As the policy iteration algorithm proceeds, better 
and better knowledge of the v,'s will result in better and better policies until an optimal policy is 
found. 

The second iteration of Howard’s algorithm leads to another control-limit policy with 
balking points n * (2. 0, 1) and an expected gain rate of g - 5.375. This policy excludes class 
two from the system. 

The socially optimal policy is found on the third iteration and is, of course, a control-limit 
policy. The balking points are - (1, 0, 1) and the expected gain rate is 5.833. Again, class 
two is excluded from the system. The stationary probabilities of the system occupying the vari¬ 
ous states under the optimal policy are <t>in n ) ** (1/3, 2/3, 0, 0). 

SUMMARY 

The single class of customers model of Naor is extended in this paper to several customer 
classes and then solved for the individual optimum policy and the social optimum policy and 
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associated gain rate. Both the individual and social optimum policies are shown to be control- 
limit policies, in addition, the socially optimal control limit for each class is shown to be no 
greater than the individually optimal control limit for the same class. The solution techniques 
are illustrated by an example. 
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ABSTRACT 


An V//67I queueing system is studied in winch the service lime required 
by it customer is dependent on the interarrival time between his arrival and 
that of Ins predecessor Assuming the iwo variables are "associated." we prove 
that the expected delay in this system is less than or equal to that of a conven¬ 
tional M/a/\ queue This conclusion has been verified via simulation by 
Mitchell and Paulson |h| |V»r a special class of dependent M/M /1 queue Their 
model is a special case of ihe one we consider here. We also study another 
modified 07/071 queue, where ihe arrival process and/or the service process 
are individually "associated “ 


1. INTRODUCTION 

The conventional G//G/I queueing model considered in most papers in the literature 
assumes that the sequences of interarrival times and service times are i.i.d. (independent and 
identically distributed) random variables. However, the independence assumption may not be 
realistic for many real world problems. The purpose of this note is to investigate the effect on 
average delay (queueing time only) of customers in systems whose interarrival and service 
times are associated, a concept of positive dependence developed by Esary, Proschan and 
Walkup {6j. 

In a recent paper, Mitchell and Paulson [9] studied via simulation an A//A//1 queue with 
the modification that a customer’s service time and the interarrival time between his arrival and 
that of his predecessor are positively correlated random variables having a bivariate exponential 
distribution. Their simulation results indicate that this type of dependency reduces the mean 
waiting time of customers as compared to the usual M/M/\ queue. Some related results also 
appeared in Conolly 12), Conolly and Hadidi 13, 4). Motivated by Mitchell and Paulson’s simu¬ 
lation results, we shall show here in Section 3 that their conclusion can be proven analytically 
under weaker conditions. In Section 2, we briefly summarize the useful concept of association 
of random variables. Another variant of G//G/1 queues with dependent interarrival times or 
service times will also be discussed in Section 4. 

2. ASSOCIATION OF RANDOM VARIABLES 

The concept of association of random variables was developed by Esary, Proschan and 
Walkup [6]. It is a very useful tool in the study of reliability systems (see Barlow and Proschan 
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111, Esary and Proschan (5}), queueing systems (see Niu 110]), and simulation (see Heidel- 
berger and (glehart (7j). 

DEFINITION 1: Random variables X = {X it .... X n ) are called associated if cov 
\f{X), £(T)] > 0 for all pairs of nondecreasing functions /and g. 

The following is a partial list of properties enjoyed by associated random variables: 


(Pl):The set consisting of a single random variable is associated. 

(P2): Nondecreasing functions of associated random variables are associated. 

(P3): Any subset of associated random variables are associated. 

(P4): If two sets of associated random variables are independent of each other, then their 
union is a set of associated random variables. 


(PS): A set of independent random variables are associated. 


The last property, P5, is a direct consequence of PI and P4. As an illustration, we will 
next show that the bivariate exponential distribution discussed in Mitchell and Paulson [9] is 
generated from a pair of associated random variables. Consider the random vector (X, Y) 
defined by 


(X. Y) 


I x,, X y, 


.i-i (-i 


where X,, i « 1, 2, ... , are i.i.d. random variables, Y n i = 1. 2.are i.i.d. random vari 

ables, and A is an integer-valued positive random variable. 


PROPOSITION!: (X, Y) is associated. 


PROOF. Let / and g be an arbitrary pair of nondecreasing functions. Conditioning on 
A =• «, we have 

cov [fix, y), gix. m 

— E Icov 


•(4 


4t t r. 


+ cov Ml. X„ X Y, |A- n\, £] 

i-i i-i 


IN 


Now, given A » the vector (*,. X „, 

random variables and hence is associated (by P5). 


Y y . Y n ) is a collection of independent 

By P2, this implies [x X„ X 


ated. Therefore, the first term above is nonnegative. The conclusion of the proposition % then 
follows by observing that both 


£ A X X n £ K, |A( - » and £ * £ *,. £ K, |A - n 

<-t i-i ,-i 


nondecreasing functions of n. 
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REMARK: When both X,, i = 1. 2.and Y n i — 1. 2.have exponential distri¬ 

butions and N is geometric, then ( X, Y) has the bivariate exponential distribution described in 
[9], 

3. AN M/G/l QUEUE WITH DEPENDENT ARRIVAL AND SERVICE 

Consider the following model of a single server queue with infinite waiting room. Denote 
by T„ the interarrival time between customers C„ and C„+\ with E{T n ) — 1/*.. Let S„ be the 
service time of customer C„ with £(£„)- My and k/y < 1. We assume ( T„. S„+|), 

n = 0, 1.is a sequence of i.i.d. random vectors and ( T „, £„+,) is associated for each n. It 

should be noted that this model includes both the conventional GI/G/l queue (where T„ and 
S„ + i are independent) and the Ml Ml 1 queue considered by Mitchell and Paulson [9j by letting 
( T „, S„ + |) to have their bivariate exponential distribution (see remark after Proposition 1). 

Denoting the delay of customer C„ by Z>„, it is well-known that 

(1) D„ + l = max[0 t D„ + S„ — T„], 
or equivalently, 

(2) D„, t - A„ = D„ + S„ - T„. 

where A„ — - min [0, D„ + - 7'„1. We will assume that the first customer arrives at time 0 

and the system is initially empty, i.e., D t = 0. The following key lemma will be needed. 

LEMMA I: cov [ D„, S„] < 0 for all n > 1. 

PROOF: The assertion is clearly true for n * 1 since £>, = 0. For n > 2, (T„_|, S„) is, 
by assumption, associated and independent of -(D„_, + S„-,). Therefore, 
(-£)„_ i - S„-|, T'.f. S „) is associated by P4. Observe that -D„ and S„ are nondecreasing 
functions of (-D„_, — S„.|, T„. S„). Hence, by P2, ( —D„, S„) is associated and conse¬ 
quently cov l D„, S„] <0. n p n 


Now, squaring both sides of (2) and taking expectations, we have 

(3) £(D„ 2 m ) + E(A,j) = E ( D,j) + £(S„ - T n ) 2 4- 2£(D„S„) - 2 E(D n T„), 


where we have used the fact that D,,.\A„ = 0. Letting n go to infinity in (3), we can cancel 
£(D„%| ) and £(D„ 2 ) from both sides assuming they are finite. Since D„ and T„ are indepen¬ 
dent and, by Lemma 1, cov \D„. Sj = E(D„S„) - E(D„) E{S„) ^ 0, the right-hand-side of 
(3) is less than or equal to 

E(S„ - T n ) 2 + 2 £(£>„) E(S„) - 2 E(D„) £( TJ. 


Noting that E(T„ - S„) = E(A„), (3) simplifies to 


(4) 


E(D„) < 


E(T n - S„M 
2 £(7"„ - S„) 


E(A 2 ) 

2 E(A„) ■ 


We are now ready for the main result in this section. 

THEOREM 1. If the arrival process is Poisson and the system is in equilibrium, then 


VOL 28. NO 3. SEPTEMBER 1981 


NAVAL RESEARCH LOGISTICS QUARTERLY 



500 


S. Nil) 


(5) 


£( 0 ,,) < 


X E(S,;) 

2(1 - X E(S„)) ' 


where the right-hand-side term is the expected delay of a conventional MIG /1 queue. 


PROOF: Note that (A„\A„ > 0) is the idle period ended by the arrival of customer C„ + |. 
Therefore, by the memoryless property of exponential distributions, ( A„\A„ > 0) is also 
exponential with rate X. Hence, 


E(At) 
2 E(A„) 


E(A?,\A„ > 0) • P[A„ > 0| 
2 E(A„\A„ > 0) ■ P{A„ > 0| 
E{A,j\A„ > 0) 

2E(A„\A„ > 0) 


- 1/X 


and from which (5) follows. 


Q.E.D. 


For arrival processes other than Poisson, the analysis of (4) becomes more difficult 
because we do not know the distribution of (A„\A„ > 0). However, various bounds for it may 
be obtained by considering special classes of arrival processes (see Marshall [8]). We shall not 
pursue this further here except for mentioning that all upper bounds for expected delay in spe¬ 
cial classes of GI/G/\ queues obtained by Marshall (8] can also be applied to our modified 
Gf/G/ 1 queues. 

4. SOME RELATED RESULTS 


In this section, we will consider another modification of Gl/Gl 1 queues which is different 
from the one discussed in Section 3. We shall assume that the sequences of service times and 
interarrival times, |S„, n > 1} and \ T„, n > I), are identically distributed and associated, i.e., 

(S|, S 2 , . . . , S„ ) and (TT 2 . T„) are associated vectors for all n > 1. Of course, this is 

again a generalization of the conventional Gl/Gl 1 queue because independent random vari¬ 
ables are associated. Now, it is easy to see that 


D„ 


max 


o. s„_, - r„-,. 


* -i 

I 



Hence, ( D„, S„ ) and ( D„ , - T„ ) are associated vectors for all n ^ 1. It follows that 
£(/)„ S„) > E(D„) E(S„) and E(D„T„) < E(D n ) E(T„). An argument similar to Section 3 
will then lead to 


( 6 ) 


£(£>„) > 


£(7; - S„) 2 
2 E(T„ ~ S„) 




Again, the second term on the right-hand-side of (6) is difficult to analyze. However, we 

have 


THEOREM 2: If \T„. n > 1| are i.i.d. exponential random variables, i.e., the arrival pro¬ 
cess is Poisson, then, under stationary conditions, we have 
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In other words, associated service times in an M/G/\ queue tend to increase the expected 
delay of a customer. This is a somewhat expected result. 

When [T„, n > I) is a renewal process (not necessarily Poisson), lower bounds for 
E(D„) may be found through (6) for several special classes of interarrival times as in Marshall 
181. As a final remark, we note that the departure process of a conventional Gl/D /1 queue 
(the service times are deterministic) is an associated process. So, (6) may be used to find a 
lower bound for the expected delay of a subsequent station. 
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ON THE OPTIMAL ALLOCATION OF CREWMEN 
ABOARD A WARSHIP 
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Den Helder. the Netherlands 

ABSTRACT 

In the temporarily isolated situation in which a warship linds itself during a 
mission, not only spare parts, but also “spare" crewmen in various categories of 
specialization must be on board. 

Mathematical models for the probabilities of mission survival for personnel and 
for personnel and materiel jointly are proposed. A practical example is worked 
out: the optimal allocation of spare crewmen to different categories of speciali¬ 
zation is calculated. 


1. INTRODUCTION 

In relation to a system on board a warship (weapons, communications etc.) a meaningful 
figure of merit is the mission reliability, defined as: "the probability that the system will operate 
in the mode for which it was designed for the duration of the mission" [1], Loosely described it 
is the probability that at the end of the planned mission the system is still in operation in an 
acceptable manner. 

Most of the equipment on board, apart from normal control, needs service, checkup, 
adjustments,'replacement of failing units, and other forms of maintenance and short-term 
repair during the mission. In terms of reliability theory, materiel and personnel are "in series": 
the unavailability of one invalidates the other. The mission reliability of a system on board is 
the joint probability that at the end of the mission the system is still in an operable state or on 
short term repairable while at the same lime at least the minimal human attendance is on hand. 

By way of precaution against nonoperation of an entire system as a result of failure of a 
vital part, spare parts are taken along. This is one of the possible forms of materiel redun¬ 
dancy. On the effect of materiel redundancy on the reliability, a reasonably well-developed 
theory is available (31. In [4l, the composition of optimal spare-part kits is dealt with. Such a 
kit, of all kits of the same cost , guarantees the highest reliability of the system that it pertains 
to. 


In the situation of temporary isolation in which a warship finds itself during a mission, 
there are no easy means to replace operators and maintenance men who have become "hors de 
combat." Aboard a modern and automated warship, a growing percentage of the crew is highly 
specialized and classified in a considerable variety of specializations, which makes it increasingly 
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difficult for one to stand in for any other outside his own category. This leads to the necessity 
for a relatively large number of "spare” crewmen in "expensive" categories to be taken along. 
As a result of this personnel redundancy it may be found that a high degree of automation does 
not automatically yield the savings in personnel costs that are generally expected. 

In view of these tendencies a modern management demands, within operational and 
financial constraints, a careful optimization in the spending of funds on "spares," where materiel 
and personnel aspects are not separable. This paper is primarily intended to be a contribution 
to the development of methods for the modeling and solution of this optimization problem. 
Related problems were treated in [5]. 

2. A MATHEMATICAL MODEL FOR THE SURVIVAL OF CREWMEN 

In this section we introduce a mathematical model for the probability for crewmen to sur¬ 
vive a mission. 

The key parameter in this model is the expected number of hits, f, that the ships incurs 
during the mission. This number is assumed to be an integer-valued Poisson-distributed ran¬ 
dom variable with mean g, so 

(2.1) Pit) = exp(-g) ■ yj-. 


The probability for one particular crewman to survive a hit is p. In consequence, the pro¬ 
bability for him to survive / hits is p\ and the probability r that he survives the mission is 

(2.2) r - £ p’Pit) = £ exp(-g) - exp(p - l)g. 


If A and B are two particular crewmen, then at each hit, their survivals are assumed to be 
independent events. This implies that the probability that both A and B survive t hits is p 7 \ 
and the probability r (2/2) that both of them survive the mission is 

(2.3) r(2/2) =* £ p : ' Pit) — £ exp(-£) ■ - exp(p 2 - l)g. 

Because of their common dependence on t, the survivals of the mission by A and by B are no 
independent events. A useful measure of interdependence is the (pairwise) correlation 
coefficient C\. orr , which is defined in the following way. Consider the two random variables x A 
and x«, where x A ** 1 if A survives the mission and x A - 0 if he does not, while xg is defined 
similarly. We now put 


COV (x A , Xg) 

/var (x 4 ) ■ var (x H ) 


(2.5) 


Eix,xg) - Eix A ) ■ Eixg) 
Eix A ) - E 2 ix A ) 
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which is found to be 
( 2 . 6 ) 

If |(/> - 1)/?| « 1, we can use the approximations 
(2.7) C corr = 1 - p 


C _ exp (p : - l)g - exp2(p - l)g 
to " ” exp ip - \)g - exp Up - 1)« ' 


-In r 
g 


( 2 . 8 ) 


/>==)- C co „ and g = 


— ~jn r 


If #and p are varied while (I - p)/?and thus r remain constant, the extreme cases are 
g = - In r and p - 0: then C\ (m - 1 
g —* oo and p —* 1: then C u>rr - 0. 


For calculations in the latter (uncorrelated) case, the following property is useful 
(2.9) exp (p‘ - 1 )j? — exp i(p - I )g -> r‘ for / — 2, 3. 


3. A MATHEMATICAL MODEL FOR THE MISSION RELIABILITY 
OF A MANNED SYSTEM 


For the case where materiel and personnel are "in series," we propose an extension of the 
mathematical model of the previous section as will be demonstrated by the following example. 

We are dealing with a system that must be manned by a crew of minimally z. At the 
beginning of the mission, w crewmen qualified to operate/maintain the system are on board. 
During the mission, the ship incurs t hits where t is integer-valued Poisson-distributed with 
mean g. At each hit, an individual has a probability of survival p, while the materiel of the sys¬ 
tem has a probability of survival a. 


Given the number of hits, /, the probability that a sufficient number of crew survives is 
(see appendix): 

,3 '" 

while the probability that the materiel survives is 
(3.2) a 1 . 


n - z| 
u 


z + u 


Hence, the probability that the materiel as well as enough personnel survive t hits is 


(3.3) 


n «--- 

n - z 

z I 

H-0 

u 


z + u 


( ap :+ "V. 


The mission reliability R ap of the system, which is the probability that the materiel as well 
as enough crew survive the mission, is obtained by multiplying (3.3) by Pit) and summing 
with respect to t. The result is 
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In the uncorrelated case, by invoking (2.9), we obtain for the mission reliability 
(3.5) 

where r p is the probability that an individual survives the mission and r„ is the probability that 
the materiel survives the mission. 

A Practical Example 

With z = 1 we have for n = 1, 2, 3, respectively, in the correlated case 
R ap = exp (ap - D* 

K P = 2 exp (ap - l)g - exp ( ap 3 - 1 )g 
R up = 3 exp (ap - \)g - 3 exp (op 2 - 1)/? + exp (ap y - l)g 
and in the uncorrelated case 
Ran ”* r a r p 

Rap “ 2 f a r p ~~ r a r p 

Ran - 3r„r p - 3 r„r‘ + r a r y . 

We vary p and g in such a way that a 

(p - l)g « constant - -0.0305 

keeping 

a - P 

which means that the probability of mission survival for an individual as well as for the materiel 
is fixed at 

r p -r a - 0.970. 

In Figure 1 we plot 1 - R ap as a function of p. In order to show the influence of the vul¬ 
nerability of the materiel, we also plot 1 - R p , which is the probability that enough crewmen 
survive the mission, regardless of materiel survival. The quantity R p derived from R ap by put¬ 
ting a - 1 or r a - 1 . 

In Table 1 we list I - R p and I - R ap as functions of n in the uncorrelated case. For 
n — oo we have 1 - R p — 0 and 1 - R up — 1 - r„. 
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* - Rap 

i for r>=1 

© 

idem 

for n = 2 

© 

idem 

for n = 3 

© 

1- R p 

for n =1 ; also 1 - R a p for 

© 

1 - Rp 

for n =2 

© 

idem 

for n = 3 



0 0.5 0.8 0.9 0.95 0,98 0.99 0.995 0.998 0.999 

-- p 

Figure I. I - R p and I - R„ as functions of p. 


TABLE I. Probabilities of Survival 
in the Uncorrelated Case 
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4. THE MISSION RELIABILITY OF A SYSTEM COMPOSED OF TWO SUBSYSTEMS 


We consider a system S composed of two subsystems S ( and S 2 . For the 
operation/maintenance of S, crewmembers are on board in two categories, 1 and 2. Crewmen 
in category 1 are competent to operate/maintain S, (but not S 2 ), while crewmen in category 2 
are competent to operate/maintain S 2 ( but not S t ). S cannot be in operation unless z\ crew¬ 
men in category 1 and z 2 in category 2 are on hand. At the beginning of the mission there are 
«i crewmen on board in category 1 and n 2 in category 2. The ship incurs t hits, where f is an 
integer-valued Poisson-distributed random variable with mean g. At each hit, the probability of 
survival of an individual is p, while the probability of survival of the materiel of S| and S 2 is a x 
and a 2 , respectively. 


The probability that after / hits the materiel of subsystem S, survives, along with a 
sufficient number of crewmen to operate/maintain it, is 


(4.1) 


"r z i 

I (-1) 1 


2) 

Z\ + «, 



(a x p’ l+u 'V 


(see appendix). 


For S 2 a similar expression can be written. After multiplying the two expressions we obtain the 
following expression for the probability that the materiel of both subsystems, along with 
enough crew, survives t hits 


(4.2) 


2l + u , 


«i - r, 

u\ 


£ (-1)"’ + "' 
u ,-0 


*1 

z 2 + u 2 


n 2 -z 2 


(a x a 2 p 


Z, + I 2 +U| + U 2 )' 


ity 

(4.3) 


By multiplying this by Pit) and summing with respect to t, we obtain the mission reliabil- 

»2 - 2j) 


Ran - 


"r- 

z, I 


l -2| 


2\ 

21 + U\ 


n, - Z\ 
Ml 


2 2 + U 2 [ M 2 


exp(fl|fl ; p' + ' + " l+ "’- l)g. 


In the uncorrelated case this changes into 

,44> itvt 

n 2 - 2 : 

«> 


n\ — Zil 

£ (-l)" l+ " 

l iTm) 


2: + u 2 


where and r aj are the probabilities that the materiel of S, and S 2 , respectively, survives the 
mission. The probability, R p , that enough crew survives the mission to operate/maintain the 
two subsystems, irrespective of materiel survival, can be derived from R ap by putting in (4.3): 
a, - oj - 1 and in (4.4): r a> - r„ } - 1. 
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For the probability R a that the materiel of both subsystems survives the mission, we have 
in the correlated case 

(4.5) R a = exp - D# 

which can be derived in the same fashion as (2.3), and which in the uncorrelated case changes 
into 

(4.6) R„ = r a/a : - 

5. SOME NUMERICAL RESULTS 

The quantities R p , R ap and R„ as dealt with in the preceding section were calculated for 
the case where 

Z] - 8 and z 2 *= 1 

while g, p, «| and n 2 were varied; a t and a 3 were varied along with psuch that always 
o, - <j : p. 

As p was varied, g was varied at the same time, keeping 
(p - l)g = constant. 

Three runs were made, with 


(p - l)j? - In 0.990, In 0.970 and In 0.900, 

respectively. 

Samples of the numerical results are given in Tables 2, 3, 4 and 5, and in Figures 2 and 3. 
In Tables 2 and 3 we list 1 - R p and 1 - R ap , respectively, as functions of (»,. n 2 ), as 
obtained at the second run, in the uncorrelated case. In Tables 4 and 5 we do the same for the 
case where p = 0.700. In Figure 2 we plot 1 - R p as a function of p for several pairs of values 
of (w|, « : ), again as obtained at the second run. The corresponding plots of 1 - R ap and 
1 - R a are given in Figure 3. 

Tables like 2, 3, 4 and 5 can be used for the determination of optimal sequences of alloca¬ 
tion. Let us suppose that the cost per crewman per mission is k t for a man in category 1 and 
k] for a man in category 2. Let us further suppose that we can afford to spend an amount K on 
the crew in the two categories. This means that we have to choose n | and n 2 under the restric¬ 
tion that 

(5.1) n ] k ] + n : Aj < K. 

For every value of A, it is easy to find a pair of values («i, « 2 ) such that under the res¬ 
triction (5.1), the lowest value of 1 - R p or 1 - R„ p is obtained. By doing this for increasing 
values of A, the optimal sequence of allocation is established. 
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TABLE 2. 1 — R p in % as a Junction of(n\, n 2 ), for 
ip - 1)# - In 0.970 and p —*1. 


II2 

»! 

H 

2 


4 

8 

HEM 

21.7 

21.6 

21.6 

9 

5.73 


2.82 

2.82 

10 

3.27 




II 

■nrai 

0.113 



12 

3.00 

0.0916 

0.00432 

0.00169 

13 

3.00 

0.0900 

0.00280 

0.185 x 10 -3 


TABLE 3. 1 - R ap in % as a Junction of 
(»i. n 2 ) for (p - l)g = In 0.970 and p — 1 


n 2 

1 

2 

3 

4 

8 

28.5 

26.3 

26.3 

26. J 

9 

11.3 

8.64 

8.56 

8.56 

10 

8.99 

6.25 

6.17 

6.17 

11 

8.75 

6.02 

5.93 

5.93 

12 

8.73 

6.00 

5.91 

5.91 

13 

8.73 

5.99 

5.91 

5.91 

14 

8.73 

5.99 

5.91 

5.91 


TABLE 4. 1 - R p in % as a Junction of (n,, n 2 ), 
for (p ~ l)g — In 0.970 andp — 0.7. 


n 2 

1 

2 

3 

4 

8 

9.28 

9.17 

9.14 

9.13 

9 

8.39 

8.01 

7.90 

7.86 

10 

7.19 

6.44 

6.22 

6.15 

11 

5.97 

4.87 

4.53 

4.43 

12 

4.97 

3.55 

3.12 

2.99 

13 

4.23 

2.60 

2.10 

1.95 

14 

3.75 

1.96 

1.41 

1.24 

15 

3.45 

1.56 

0.985 

0.806 

16 

3.27 

1.33 

0.728 

0.542 

17 

3.16 

1.19 

0.576 

0.385 

18 

3.10 

1.10 

0.485 

0.290 

19 

3.05 

1.02 

0.394 

0.195 
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TABLE 5. 1 - R f in % as a Junction of(n\, n 2 ), 
for (p - 1)# - In 0.970 and p - 0.7. 


«2 

"1 

1 

2 

3 

4 

8 

9.47 

9.42 

9.40 

9.40 

9 

9.04 

8.85 

8.80 

8.78 

10 

8.4S 

8.09 

7.98 

7.94 

11 

7.86 

7.32 

7.15 

7.10 

12 

7.37 

6.68 

6.47 

6.41 

13 

7.01 

6.22 

5.98 

5.91 

14 

6.78 

5.92 

5.65 

5.57 

15 

6.64 

5.73 

5.45 

5.37 

16 

6.56 

5.62 

5.34 

5.25 

17 

6.51 

5.56 

5.27 

5.18 



Figure 2. I - R p vs. p for several values of In,. n 2 ), for (p - 1) * - In 0.970 
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0 0.5 0.8 0.9 0.95 0.98 0.99 0.995 0.998 0.999 

-- p 

Figure 3. I - R a and 1 - R ap vs. p for several values of (n,. n 2 ), for (/> - 1) * - In 0.970 


Results are depicted in Figures 4 and 5, which show allocation sequences optimizing 
I - R p and 1 - R ap , respectively. In both of these figures, the black dots refer to the situation 
where 


*1 “ *2 

whereas the open dots apply to the situation where 
: k 2 - 1 : 2. 

In the former case the sequence is discontinued after the first step where 1 - R p or 
1 - R ap is reduced by less than 0.5%; in the latter case the sequence is continued up to the 
same final value. 

The effect of the first four optimal allocation steps (that is to say, optimal for the case 
Jk, - k 2 ) on the numerical values of 1 - R p and l - R up is shown in Tables 6 and 7, respec¬ 
tively. The parameters in these tables are the same as in Figures 4 and 5. In Table 7 we also 
give the limit values 1 - R u of 1 - R ap as n t and approach infinity. 
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SOME OBSERVATIONS ON THE RESULTS: 

1. Comparing situations where p is equal to or close to 1 (so C corr = 0) with situations 
where p is smaller (so C corr is larger), we find that in the former case the mission 
reliability with (ii|, n 2 ) - (8, 1) is smaller, but the addition of spare crewmen is 
more effectual. 

2. Comparing the optimal allocation sequences in the various cases, we find that these 
sequences are not too strongly dependent on p, g, a t a 2 or k 2 /k { . In all cases n\/tt 2 
tends to become smaller than z\h 2 . In other words we recognize a tendency to allo¬ 
cate a relatively large portion of the spares to the smaller number. 

3. Comparing the allocation sequences for a particular value of p, while In (p - l)g is 
varied, we recognize the tendency of njn 2 to approach Z\/z 2 as exp (p - l)g 
decreases: if the risk of the mission increases, the extra allocations go primarily to 
the larger number. 

4. There is a significant difference between spare parts in a spare part kit and "spare” 
crewmen aboard a warship. Only parts in active state can fail, whereas parts in 
storage ("cold standby") will not get out of order. A "spare" crewman, on the other 
hand, is vulnerable as soon as the mission is begun (in terms of reliability theory, he 
is in "hot standby"). Despite this difference, however, the results obtained here are 
in accordance with the rules which were found to apply to optimal spare part kits (4], 

5. Comparing R n with R ap under the same circumstances, we find that the addition of 
spare crewmen has less effect on R ap than on R n . Since the difference between R ap 
and R p is caused by the vulnerability of the materiel, this illustrates that the addition 
of spare crewmen in less effectual as the materiel is more vulnerable. Results like 
the ones we have obtained here could be useful for making the choice between 
spending available funds on spare crew or on improving the mission reliability of the 
equipment. 

6. CONCLUDING REMARKS 

In this paper, mathematical models for the mission reliabilities of personnel, and person¬ 
nel and materiel jointly, were presented. An example was worked out, where the optimal allo¬ 
cation of crewmen in different categories of specialization was calculated. Despite its simplicity, 
this example is indicative of the results that can be obtained by application of the same tech¬ 
niques to more complex configurations. 

From our calculations we have learned that the mission reliability of a manned system not 
only depends on the probabilities of survival of individual crewmen and separate pieces of 
equipment, but equally on the correlations between survivals. This implies that efforts must be 
made to gain insight into the statistical characteristics of risks that personnel and equipment are 
exposed to during a mission. 
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APPENDIX 

Consider n possible events e t , e 2 . e„ which are symmetrical in the sense that 

(A.la) P(e,) = Pie : ) P(e n ) 

(A.lb) Pie\e 2 ) — Pie |P_0 —_** Pie,, \ e„) 

(A.lc) Pie ifs?,.) “ Pie^e^e^) —.— Pie,, : e,,., e„) 


Then the following property is known [21. The probability Pi ^ 2 , n) that at least revents 
happen is 


(A. 2) 


Pi> z, n) = 


A I <-<>' 



—7— tt(z + tt) 
2 + U 


where it (2 4- w) is the probability (hat e\, e 2 . e. + „ all happen, irrespective of whether 

the others happen or not. 
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STOCHASTIC DUELS WITH DISPLACEMENTS (SUPPRESSION)* 
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ABSTRACT 

The stochastic duel is extended to include the possibility of a near-miss on 
each round tired, which causes the opponent to displace. During displacement, 
the displacing contestant cannot return the tire but is still a target for his op¬ 
ponent. An alternative interpretation of this model is to consider the displace¬ 
ment time as the time a contestant's tire is suppressed by his opponent's tire 
and that he does not move, but merely ceases fire temporarily. All limes are 
exponentially distributed. 


1. INTRODUCTION 

This paper continues the development of the Theory of Stochastic Duels. Although com¬ 
bat is a very complex process, microscopic models of this type help to develop important 
insights into the more complex processes and are also useful in weapons systems analyses, 
among other things. 

Some earlier work has considered mobility by incorporating varying projectile time-of- 
flight, see Ancker 12] and Jaiswal and Bhashyam (51, or by varying hit probability, see Williams 
(81 and Bhashyam and Singh 14} and Ancker [1]. Some very simple models, where the contes¬ 
tants displace, are provided in Ancker and Williams (31 and Schoderbeck (61 and (71. In this 
paper we consider a more general situation of the latter type. 

In general, most earlier studies of stochastic duels have not considered an interaction 
between the two contestants, A and B. That is to say, the same results would have been 
effected if each fired separately at his own target in two different localities and they subse¬ 
quently compared their times to score a kill, having previously agreed that the quicker of the 
two would be the winner. We shall now consider a duel where A' s behavior is contingent upon 
B s, and vice versa. 


•This study was partially supported by the Department of the Army, U.S. Army TRADOC Systems Analysis Activity, 
White Sands Missile Range, New Mexico. 
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2. THE MODEL 


We assume, as usual, that both duelists begin to load and fire simultaneously, but we now 
add the possibility that one or the other may score a near-miss, the effect of which on a duelist 
is to make him move to a new firing position. We may imagine that if he did not move, he 
would immediately be killed, since his opponent has now gotten his range. During his displace¬ 
ment time, he is subject to fire from his opponent but cannot return it. We assume that the 
probabilities of a near-miss and a kill are the same from round to round, and that once a duelist 
has displaced because of a near-miss, he proceeds to load and fire as before. His displacement 
and firing times are random variables whose probability density functions are known and are 
not necessarily the same. 

An alternate interpretation that may be useful is as follows: Upon receiving a near-miss, 
the duelist merely seeks cover in his present position and ceases fire for a period of lime equal 
to the corresponding displacement time, i.e., merely interpret displacement time as fire- 
suppression time, and the model is a fire-suppression model as it stands. 

3. ANALYSIS 


Our first step is to eliminate from consideration all complete misses on both sides. We 
are thus left with a series of near-misses that form, so to speak, a succession of turning points 
on which the duel hinges. It does not matter which of the duelists scores a near-miss, since it 
still interrupts the course of the duel. We thus make a list of when the near-misses occurred 
and who scored them. Ultimately, the duel ends on a near-miss which was actually a kill. 


We introduce the following notation. Let 

p AA - the conditional probability that A scores the 

next near-miss, given that A scored the last one. 


( 1 ) 


p BA - the conditional probability that B scores the 

next near-miss, given that A scored the last one, 
and similarly for p AB and p BB , Then, we plainly have 


( 2 ) 

Next, let 

(3) 

and 


Paa + Pba ” Pab + Pbb “ 1- 

p A - the unconditional probability that A scores 
a near-miss on any round, 


k A - the conditional probability that A scores 
a kill, given a near-miss, 
that is, on any near-miss, a kill may or may not have occurred, and 
k A p A - the joint probability of a near-miss and a 
kill, or just simply a kill. 


(4) 


(5) 
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Also, let 

(6) 

and 

( 7 ) 


q A “ 1 - Pa “ the probability that A misses 
entirely on any round 

l A — I - k A — the conditional probability that A does 
not score a kill, given a near-miss. 


Of course, there is a similar notation for B. We may remark here, that it will be important in 
some future research to allow these probabilities, i.e., (3), (4), (6) and (7), to be different in 
the suppressed and nonsuppressed states. 


(8) 


We further let 

Poa 

P()H 


the probability that A scores the first near-miss 
the probability that B scores the first near-miss 


so that 


(9) 

and we call 

( 10 ) 


Poa + Poh “ 1 

| P„(A ) = the probability that A kills, for the 
first time, on near-miss number n 


PJB) = the probability that B kills, for the 
first time, on near-miss number n 
Then, the probability, P(/0, that A wins the duel is 


( 11 ) 


P(A ) — £ P„(A ) . 
and, similarly for B 

P(B) - £ P„(B) . 

n?\ 


Also, P(A) + P(B) - 1. Clearly, when n - l, we have 
(12) PM)-Po A k A and P\(B) - p„ B k B . 


Now, the probability that A scored a near miss on round n — 1, but did not kill, is given 
by 

T, P -' U) 

since P„ t {A) includes a factor k A for the kill on round n - 1, which factor must be replaced 
by I A if there was no kill. Thus, 
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(13) 


P„(A) = k A 


PA4~P n P„ ,(B) 


provides the two near-miss situations on a given round that can lead to a kill by A on the next 
round. Hence, 


(14) 


P„(A) = l A p 4A P n ,(A) + l„ p 4B 



(B), 


and similarly, 

(15) P„(B) - l H pffff P„^(B) + l A p HA P„~\(A). 


Summing both sides of these simultaneous difference equations, from 2 to °°, and making 
use of Equations (11) and (12), we find that 

(16) P(A) - p ()4 k 4 = l 4 p 44 P(A) + Iff p 4li y 1 P(B), 

Afl 

and 

(17) P(B) - Poffkff = Iff Pffff p(B) + l 4 p BA — P(A). 

k a 


When we solve these two equations simultaneously for P(A ), we obtain 


(18) 


P(A) - 


k A (pp 4 kff + lffP A H) 

— Ia Paa kff + Iff Pab k 4 


with a corresponding expression for P(B). By using Notations (1), (2), and (7), it is easily 
shown that the denominators of P(A) and P(B) are equal. 


Now, adding the numerator of P(A) to that of P(B), we find 

k A kffp OA + k A Iff p 4 ff + k A kg Poff + kff l A pg A , 

which, by using Equations (2) and (9), is equal to 

k 4 kff + k 4 Iff p 4 b + k B l A - kg l A p 4A . 

And now, using Equation (7), this is equal to 
kn + k 4 Iff p AH - kff l A p AA , 

which is equal to the common denominator of P(A) and P(B), thus checking that P(A) + 
P(B) = 1. 


Now, assume that 4’s firing time is exponentially distributed with mean \/r A (r A is 4's 
rate of fire), and his displacement time is also an exponential with mean 8^, and similarly for 
B. Then, by exactly the same reasoning as used in the fundamental duel, 191, to arrive at the 
first kill, t he probability that A makes the first near-miss, p OA , is 


(19) 


POA 


P A r A 


PA r A + PffTff 

Also, from Notation (1), 

(20) p AA - P\T a <:T b + D a ] 
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T a — random variable time between A ’s near-misses, 

T b = time between B' s near-misses, and 
D b = fi’s time to displace. 

This expression accounts for the fact that after A scores a near-miss, he proceeds to fire 
again, whereas B must first displace and then fire. And, as we have already seen, when the 
probability of the event (near-miss) is p A and the firing time is exponential with mean \/r A , 
then the time to score a near-miss is exponentially distributed with mean 1 /p A r A . Again, we 
may use the solution of the fundamental duel to obtain this probability, p AA , by noting that the 
characteristic function of T B + D s is simply the product of their individual characteristic func¬ 
tions. Hence, 

_ _1_ /•“-« _ PA r iPn r B du _ 

5 Paa 2iri (p A r A + iu) (p B r B - iu) (\ - ih B u) 

and, using residue theory, we have 

_ Pa'aU + P\r A h B 4- p B r B h B ) 

Paa ( p 4 r A + p B r H ) (l + p A r A S H ) ’ 

and similarly, 

= _ Pa t a _ 

PAH (PA r 4 + Ph rj <1 + PBr B b A ) ‘ 

When we substitute these results into Equation (18), we obtain, after a little algebra, 

(24) /M.t) k APAriU + Pa'a^h) (> + k H p B r B b A ) 

IiaPa'a^ + p A r A h B ) (1 + k B p B r B h A ) 

+ W»t«( 1 + p H r H b A > (1 + k A p 4 r A b H ) 

4. CONCLUSION 

It is interesting to note that if k A - k B ~ 1, then all near-misses are really kills, and Equa¬ 
tion (24) reduces to the solution for the fundamental duel with exponential firing times, 19], 
and the same result occurs if b 4 = b B * 0 (zero displacement), as it should. 

The expression in Equation (24) is unwieldy, since it contains eight parameters. If we 
define 

Ph “ PH r H&H’ 

P 4 * p A r A b A , and 
D - h„lh A 

then Equation (24) becomes 

_ Pk A P A (P + k B P B ) (1 + PP A ) _ 

(25) PiA) - k4Dp {D + kff p H )(\ + 0P A ) + k a P B iD + P B ) (1 + k A DP A ) 

which reduces the parameters to only five. 
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